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(54) Data transfer, synchronising applications, and loyv latency networks 



(57) Asynchronous network interface and method of 
synchronisation between two applications on different 
computers is provided. The network interface contains 
snooping hardware which can be programmed to con- 
tain triggering values comprising either addresses, ad- 
dress ranges or other data which are to be matched. 
These data are termed "trip wires". Once programmed, 
the interface monitors the data stream, including ad- 
dress data, passing through the interface for addresses 
and data which match the trip wires which have been 
set. On a match, the snooping hardware can generate 



iritegjjpt, increment event counters, or perform some 
oi^e£ application -specified action. This snooping hard- 
ware!; jjs preferably based upon Con tent- Addressable 

.^^|e Invention thus provides in-band synchronisa- 
tion^ synchronisation primitives which are pro- 
gra^ri^able by user level applications, while still deliver- 
[npjijigh bandwidth and low latency. The programming 
p^ihi^fsynchronisation primitives can be made by the 
sending and receiving applications independently of 




eapnvpther and no synchronisation information is re- 
'qy^^|to traverse the network. 
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Description 

[0001] This invention, in its various aspects, relates 
to the field of asynchronous networking, and specifically 
to: a memory mapped network interface; a method of 
synchronising between a sending application, running 
on a first computer, and a receiving application, running 
on a second computer, the computers each having a 
memory mapped network interface; a communication 
protocol; and a computer network. This invention also 
relates to data transfer and to synchronising applica- 
tions. 

[0002] Due to a number of reasons, traditional net- 
works, such as Gigabit Ethernet, ATM, etc., have not 
been able to deliver high bandwidth and low latency to 
applications that require them. A traditional network is 
shown in Fig. 1 . To move data from computer 200 to 
another computer 201 over a network, the Central 
Processing Unit (CPU) 202 writes data from memory 
204 through its system controller 206 to its Network In- 
terface Card (NIC) 21 0. Alternatively, data may be trans- 
ferred to the NIC 210 using Direct Memory Access 
(DMA) hardware 21 2 or 21 4. The N IC 21 0 takes the data 
and forms network packets 216, which contain enough 
information to allow them to be routed across the net- 
work 21 8 to computer system 201 . 
[0003] When a network packet arrives at the NIC 211 , 
it must be demultiplexed to determine where the data 
needs to be placed. In traditional networks this must be 
done by the operating system. The incoming packet 
therefore generates an interrupt 207, which causes soft- 
ware, a device driver in operating system 209, to run. 
The device driver examines the header information of 
each incoming network packet 216 and determines the 
correct location in memory 205, for data contained with- 
in the network packet. The data is transferred into mem- 
ory using the CPU 203 or DMA hardware (not shown). 
The driver may then request that operating system 209 
reschedule any application process that is blocked wait- 
ing for this data to arrive. Thus there is a direct sequence 
from the arrival of incoming packets to the scheduling 
of the receiving application. These networks therefore 
provide implicit synchronisation between sending and 
receiving applications and are called synchronous net- 
works. 

[0004] It is difficult to achieve optimum performance 
using modern synchronous network hardware. One rea- 
son is that the number of interrupts that have to be proc- 
essed increases as packets are transmitted at a higher 
rate. Each interrupt requires that the operating system 
is invoked and software is executed for each packet. 
Such overheads both increase latency and the data 
transfer size threshold at which the maximum network 
bandwidth is achieved. 

[0005] These observations have led to the develop- 
ment of asynchronous networks. In asynchronous net- 
works, the final memory location within the receiving 
computer for received data can be computed by the re- 
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ceMhg NIC from the header Information of a received 
network packet. This computation can be done without 
th^aSS of the operating system. 
[0006] Hence, in asynchronous networks there is no 
ne§g!tp generate a system interrupt on the arrival of in- 
cSrtirjg data packets. Asynchronous networks therefore 
hayeithe potential of delivering high bandwidth and low 
jate'r&y; much greater than synchronous networks. 
Ttfo^rtual Interface Architecture (VIA) is emerging as 
ajstandard for asynchronous networking. 
[Q00[7]':. Memory-mapped networks are one example of 
asyncHronous networks. An early computer network us- 
Ingfmemory mapping Is described in US patent No. 
4r393;443. 

[0008] A memory-mapped network Is shown in Fig. 2. 
Agglteation 222 running on Computer 220 would like to 
cpjmjTiunicate with application 223 running on Computer 
22#ysing network 224. A portion of the application 222's 
memcj/y address space is mapped using the computer 
20 220|syirtual memory system onto a memory aperture of 
th^yjC 226 as shown by the application's page-tables 
228;;{these page-tables and their use is well known in 
tftejgjt)! Likewise, a portion of application 223's memory 
adcjre^ space is mapped using computer 221's virtual 
25 m<|njojy system onto a memory aperture of the NIC 229 
.MStagj^i® application 223's page-tables 231. Software 



30 



15 



35 



40 



45 



55 



Isjusually required to create these mappings, but once 
' trteyljiave been made, data transfer to and from a re- 
r^otejjnachine can be achieved using a CPU read or 
w|j|^instruction to a mapped virtual memory address. 
[0009] If application 222 were to issue a number of 
pp^ssor write instructions to this part of its address 
: s||,q^the virtual memory and I/O controllers of compu- 
terdom ensure that these write instructions are cap- 
tu^iffby the memory aperture of the NIC 226. NIC 226, 
determines the address of the destination computer 221 
ahtl|ihe address of the remote memory aperture 225 
wtttyplthat computer. Some combination of this address 
inJ^rTnation can be regarded as the network address, 
w|fiid]j.is the target of the write. 
[00^,0] All the aperture mappings and network ad- 
dress translations are calculated at the time that the con- 
nection between the address spaces of computers 220 
and 221 is made. The process of address lookups and 
translations at each stage in the system can be carried 
oufeysjng hardware. ' 

[00^1 ] ( ' After receiving a write, NIC 226 creates net- 
wbr&packets using its packetisation engine 230. These 
papists are forwarded to the destination computer 221 . 
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/ratj^destination, the memory aperture addresses of 
the|[r|cpming packets are remapped by the packet han- 
d^onto physical memory locations 227. The destina- 
tio^wJ^lC 229 then writes the incoming data to these 
p|jy^l^l memory locations 227. This physical memory 
hgs^lso been mapped at connection set-up time into 
tne^p'tiress space of application 223. Hence application 
2^^able, using page-tables 231 and the virtual mem- 
pry&yjstem, to access the data using processor read and 
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write operations. 

[0012] Commercial equipment for building memory- 
mapped networks is available from a number of ven- 
dors, including Dolphin Interconnect Solutions. Industry 
standards, such as Scalable Coherent Interface (SCI) s 
(IEEE Standard 1596-1992), have been defined for 
building memory mapped networks, and implementa- 
tions to the standards are currently available. 
[001 3] SCI is an example of an asynchronous network 
standard, which provides poor facilities for synch ronisa- io 
tion at the time of data reception. A network using SCI 
is disclosed in US Patent No. 5,81 9,075. Figure 3 shows 
an example of an SCI- like network, where application 
242 on computer 240 would like to communicate with 
application 243 on computer 241 . Let us suppose that 15 
application 243 has blocked waiting for the data. Appli- 
cation 242 transmits data using the methods described 
above. After sendingthe data, application 242 must then 
construct a synchronisation packet in local memory, and 
program the event generator 244, in NIC 246, to send 20 
the synchronisation packet 248, to the destination node. 
[0014] On receiving synchronisation packet 248, the 
NIC 245 on computer 241, invokes its event handler 
247, which generates an interrupt 249 allowing the op- 
erating system 248 to determine that application 243 is 25 
blocked and should be woken up. This is called out-of- 
band synchronisation since the synchronisation packet 
must be treated as a separate and distinct entity and not 
as part of the data stream. Out-of-band synchronisation 
greatly reduces the potential of memory-mapped net- 30 
works to provide high bandwidth and low latency. 
[0015] In other existing asynchronous networks, such 
as the newly emerging Virtual Interface Architecture 
(VIA) standard and the forthcoming Next Generation In- 
put/Output (NGIO) standard, some support is provided 35 
for synchronisation. A NIC will raise a hardware interrupt 
when some data has arrived. However, the interrupt 
does not identify the recipient of the data, instead only 
indicates that some data has arrived for some commu- 
nicating end-point. *o 
[0016] While delivery of data can be achieved solely 
by hardware, the software task of scheduling between 
a large number of applications, each handling received 
data, becomes difficult to achieve. Software, known as 
a device driver, is required to examine a large number 45 
of memory locations to determine which applications 
have received data. Itmustthen notify such applications 
that data has been delivered to them. This might include 
a reschedule request to the operating system f orthe rel- 
evant applications. so 
[0017] Other known data transfer techniques are dis- 
closed in EP 0 600 683, EP 0 359 137, EP 0 029 800, 
US 5 768 259, US 5 550 808 and JP 600211559. : 
[0018] According to a first aspect of the invention, 
there is provided a method as defined in the appended 55 
claim 1 . 

[0019] According to a second aspect of the invention, 
there is provided an interface as defined in the append- 




ed?claim 16. 

[0020] According to a third aspect of the invention; 
there is provided a method as defined in the appended 
cj|gi>6. 

fl)021] According to a fourth aspect of the invention, 
uSere/fs provided a method as defined in the appended 
cljjgfj43. 

[0022], According to a fifth aspect of the invention, 
themis provided a network as defined in the appended 
clajm.63. 

. v a** 

[0023] According to a sixth aspect of the invention, 
th^|^ls provided a method as defined in the appended 
cfa^f^7. 

[0^4],' Preferred embodiments of the invention are 
defied in the other appended claims. 
[0025]: It is thus possible to address the synchronisa- 
tion']) rpblem for memory mapped network interfaces. An 
embodiment of the invention uses a network interface, 
CQCifeining snooping hardware which can be pro- 
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gj^njrried to contain triggering values comprising either 
adpir|Sses, address ranges, or other data which are to 
b||j^tehed. These data are termed "Tripwires". Once 
p|og|ammed, the interface monitors the data stream, in- 
clMjing address data, passing through the interface for 
a^r^|ses and data which match the Tripwires which 
h^e^been set. On a match, the snooping hardware can 
generate interrupts or increment event counters, or per- 
fqrjri^some other application specified action. This 
srj|oplng hardware Is preferably based upon Content 
Addressable Memory (CAM). References herein to the 
"data Stream" refer to the stream of data words being 
transferred and to the address data accompanying 

[0p26J It is also possible to provide in -band synchro- 
njsa^on by using synchronisation primitives which are 
p^gl^mmable by user level applications, while still de- 
liyej|nig high bandwidth and low latency. The program- 
rrjirj^oT the synchronisation primitives can be made by 
th^ending and receiving applications independently of 
eacifejbther and no synchronisation information is re- 
p,gjpd»tp traverse the network. 
[00^]"; A number of different interfaces between the 
:ni|^^rk Interface and an application can be supported. 
Tj[i Is ^interfaces include VIA and the forthcoming Next 
(^erjation Input/Output (NGIO) standard. An interface 
car|^ie chosen to best match an application's require- 
ments- and changed as its requirements change. The 
netvy^rk interface can support a number of such inter- 
faceasimultaneously. 

[QQ28] The Tripwire facility supports the monitoring of 
qujgomg as well as incoming data streams. These Trip- 
wlre^'ban be used to inform a sending application that 
its^KflA send operations have completed or are about 
Wjc^pplete. ' 

[0029] Memory-Mapped network interfaces also have 
t&j^otential to be used for communication between 
lip^are entities. This is because memory mapped net- 
w!ap^terfaces are able to pass arbitrary memory bus 



2.0.1 .4 




EP 1 302 8533fc2> 



6 



cycles overthe network. As shown in Fig. 4, it is possible 
to set up a memory aperture 254, in the NIC 252 of Com- 
puter 250, which is directiy mapped via NIC 259, onto 
an address region 257 of the I/O bus 253 of passive 
backplane 251 . 5 
[0030] Using existing memory mapped interfaces, 
such as DEC Memory Channel or Dolphin SCI, an ap- 
plication running on Computer 250, which requires use 
of the hardware device 255, would require a (usually 
software) process to interface between itself and the 10 
Network Interface card (NIC) 252. This is because the 
NIC 252, would not appear at the hardware level in com- 
puter 250 as an Instance of the remote hardware device 
255, but instead as a network card which has a memory 
aperture 254 mapped onto the hardware device. 15 
[0031 ] It is also possible to provide an interface which 
can be programmed to present the same hardware in- 
terface as the remote hardware device 255, and so ap- 
pear at the hardware level in computer 250 to be an in- 
stance of the remote hardware device, if the network 20 
card 252 were an interface according to the present in- 
vention, so programmed, the remote hardware device 
255 would appear as physically located within computer 
250, in a manner transparent to all software. The hard- 
ware device 255, is able to by physically located both at 25 
the remote end of a dedicated link, or over a general 
network. Both general networking activity and remote 
hardware communication can be supported simultane- 
ously on a single network card. 

[0032] A link-level communication protocol can be 30 
used to support cut-through routing and forwarding. 
There is no need for an entire packet to arrive at a NIC, 
or any other network entity supporting the communica- 
tion protocol, before data transmission can be started 
on an outgoing link. Large bursts of data can be handled 35 
effectively without the need for a small physical network 
packet size such as that employed by an ATM network, 
it being possible to dynamically stop and restart a burst 
and regenerate all address information using hardware. 
[0033] A preferred embodiment of the various aspects 40 
of the invention will now be described with reference to 
the drawings in which: 

Figure 5 shows two or more computers connected 
by an embodiment of the present invention, using 45 
Network Interface Cards (NICs); 
Figure 6 shows in detail the various functional 
blocks comprising the NICs of Figure 5; 
Figure 7 shows the functional blocks of the NIC 
loyed within a Field Programmable Gate Array (FP- 50 
GA); i; 
Figures 8 and 8e shows the communication proto- 
col used in one embodiment of the invention; 
Figure 9 shows schematically hardware communi- 
cation according to an embodiment of the invention; 55 
Figure 1 0 shows schematically a circular buffer ab- 
straction according to one embodiment of the inven- 
tion; 



jifigure 11 shows schematically the system support 
t|or discrete message communication using circular 
puffers; 

figure 12 shows a client-server interaction accord- 
ling to an embodiment of the invention; 
figure 13 shows how the system of the present in- 
dention can support VIA; 

jFigure 14 shows outgoing stream synchronisation 
^£ccording to an embodiment of the present inven- 

I^Sgure 15 shows a client-server interaction accord- 
..'Mlngto an embodiment of the invention using a hard- 
;i'|pyare data source; 

|^|(gure 1 6 shows an apparatus for synchronising an 
{ |r||nd-point application and constituting an embodi- 
/.S&fent of the invention; 

- il M Figure 1 7 shows another apparatus for synch ronis- 
'-^ng an end-point application and constituting an em- 
; .Jpifebdiment of the invention; 
^^^jgures 18 to 23 show examples of actions which 
^fefiay be performed by the apparatuses of Figures 



/.'l^^and 17; 
^^niplied addresses; 

■'^^^igure 25 illustrates an interruption in forwarding a 
M^urst of the type shown in Figure 24; 
'Mpigure 26 illustrates forwarding of the rest of the 

l^gjgure 27 illustrates coalescing of two data bursts; 
jC^pigure 28 illustrates "transparent" communication 
Mjoyer a network between an application running on 
t^|jComputer and remote hardware; and 
' ^rtigure 29 illustrates applications of various tripwires 
Indifferent locations in a computer. 



[0pa^] f Referring to Figure 5, computers 1, 2 use the 
p'^^pt invention to exchange data. A plurality of other 
computers such as 3, may participate in the data ex- 
change if connected via optional network switch 4. 
[Oj^^i Each computer 1 , 2 is composed of a micro- 
p|o^essor central processing unit 5,57, memory 6,60, 
lpca|cache memory 7,57, and system controller 8,58. 
^e^ystem controller 8,5B interacts with its microproc- 
essor^^ to allow the microprocessor to exchange da- 
tajwitft devices attached to I/O bus 9. Attached to I/O 
b|^9f59 are standard peripherals, such as a video 
a]daipter 1 0. Also attached to I/O bus 9,59 is one or more 
n|^rofk interfaces, in the form of NICS 11,56 which rep- 
resent an embodiment of this invention. In computers 1, 



j/O bus is a standard PCI bus conforming to PCI 
LqcMBus Specification, Rev. 2.1, although any other 
busVapable of supporting bus master operations can 
bejusad with suitable modification of System Controller 
peripherals, such as video card 10, and the interface to 

[Q03j|], Referring to Figure 6, each NIC comprises a 
memory 18, 19, 20 for storing triggering values, a re- 
cepe^»15 for receiving a data stream, a comparator for 
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comparing part of the data stream with the triggering val- 
ues and a memory 23 for storing information which will 
identify matched triggering values. More specifically, in 
the preferred embodiment each NIC 56, 1 1 is composed 
of a PCI to Local Bus bridge 1 2, a control Field Program- 5 
mable Gate Array (FPGA) 13, transmit (Tx) serialiser 1 4, 
fibre-optic transceiver 15, receive (Rx) de-serialiser 16, 
address multiplexer and latch 1 7, CAM array 18,19, 20, 
boot ROMs 21 and 22, static RAM 23, FLASH ROM 24, 
and clock generator and buffer 25, 26. Figure 6 also to 
shows examples of known chips which could be used 
for each component, for example boot ROM 21 could 
be an Altera EPC1 chip. 

[0037] Referring to Figure 7, FPGA 13 is comprised 
of functional blocks 27-62. The working of the blocks will is 
be explained by reference to typical data flows. 
[0038] Operation of NIC 11 begins by computer 1 be- 
ing started or reset. This operation causes the contents 
of boot ROM 21 to be loaded into FPGA 1 3 thereby pro- 
gramming the FPGA and, in turn, causing state ma- 20 
chines 28, 37, 40, 43, 45, 46 and 47 to be reset. 
[0039] Clock generator 25 begins running and pro- 
vides a stable clock forthe Tx serialiser 1 4. Clock buffer/ 
divider 26 provides suitable clocks for the rest of the sys- 
tem, Serialiser 1 4 and de-serialiser 1 6 are reset and re- 25 
main in a reset condition until communication with an- 
other node is established and a satisfactory receive 
clock is regenerated by de-serialiser 1 6. 
[0040] PCI bridge 1 2 is also reset and loaded with the 
contents of boot ROM 22. Bridge 12 can convert (and 30 
re-convert at the target end) memory access cycles into 
I/O cycles and support legacy memory apertures, and 
as the rest of the NIC supports byte-enabled (byte-wide 
as well as word-wide) transfers, ROM 22 can be loaded 
with any PCI configuration space information, and can 35 
thus emulate any desired PCI card transparently to mi- 
croprocessors. 

[0041] Immediately after reset, FLASH control state 
machine 47 runs and executes a simple microcode se- 
quence stored in FLASH memory 24. Typically this al- <*o 
lows the configuration space of another card such as 69 
in Figure 9 to be read, and additional information to be 
programmed into bridge 1 2. Programming of the FLASH 
memory is also handled by state machine 47 in conjunc- 
tion with bridge 1 2. 45 
[0042] Data transfer could in principle commence at 
this point, but arbiter 40 is barred from granting bus ac- 
cess to Master state machine 37 until a status bit has 
been set in one of the internal registers 49, This allows 
software to set up the Tripwires during the initialisation 50 
stage. 

[0043] Writes from computer 1 to computer 2 take 
place in the following manner. Microprocessor 5 writes 
one or more words to an address location defined by 
system controller 8 to lie within NIC 1 1 's address space, ss 
PCI to local bus bridge 12 captures these writes and 
turns them into local bus protocol (discussed elsewhere 
in this document). If the writes are within the portion of 




address space determined to be within the local con- 
aperture of the NIC by register decode 48, then the 
w^es^take place locally to the Content Addressable 
Menr^ry appropriate register, (CAM), Static RAM 
(^R/^) or FLASH memory area. Otherwise target state 
rn^hine 28 claims the cycles and forwards them to pro- 
to^pffericoder 29. 

P 0,4g] At the protocol encoder, byte-enable, parity da- 
tamhd control information are added first to an address 
and then to each word to be transferred in a burst, with 
ajcofitjrol bit marking the beginning of the burst and pos- 
slbiwalso a control bit marking the end of the burst. The 
,comrol bit marking the beginning of the burst indicates 
triaj^ddress data forming the header of the data burst 
comprises the first "data" word of the burst. Xon/Xoff- 
sty^rpanagement bits from block 31 are also added 
hej^e| Jhis protocol, specific to the serialiser 14 and de- 
senaliser 16 is also discussed elsewhere in this docu- 
men*"- 



[0P ( 45]- : Data is fed on from encoder 29 to output mul- 
tigjejef 30, reducing the pin count for FPGA 13 and 



.mating the bus width provided by serialiser 14. Seri- 
ai^etil;4 converts a 23-bit parallel data stream at 62MHz 
t(0\l||bit data stream at approximately 1 .5Gbit/s; this is 
cjp|w|rted to an optical signal by transceiver 1 5 and car- 
ried over a fibre-optic link to a corresponding transceiver 
'iS^KijC 56, part of computer 2. It should be noted that 
othe ( r physical layers and protocols are possible and do 
nbt^ljmlt the scope of the invention. 
[0046]; In NIC 56, the reconstructed digital signal is 
clock-recovered and de-serialised to 62MHz by block 
j16||1pck 32 expands the recovered 23 bits to 46 bits, 
reversing the action of block 30. Protocol decoder 33 
cpecks that the incoming words have suitable sequenc- 
es^o^pntrol bits. If so, it passes address/data streams 
irjt^cjitirimand FIFO 34. If the streams have errors, they 
a|^|oJa^sed into error FIFO 35; master state machine 37 
(shopped; and an interrupt is raised on microprocessor 
5||jby,block 53. Software is then used to decipher the 
in^gjfng stream until a correct sequence is found, 
w|eretipon state machine 37 is restarted. 
V&0t; s When a stream arrives at the head of FIFO 34, 
majt^rjstate machine 37 requests access to local bus 
5|^r|jm arbiter 40. When granted, it passes first the ad- 
ftresj^then the following data onto local bus 55. Bridge 
1^I&acts to address/data stream by requesting ac- 
cessito I/O bus 59 from system controller 58. When 
granted, it writes the required data into memory 60. 
[0048] Reads of computer 2's memory 60 initiated by 
computer 1 take place in a similar manner. However, 
sjat$*machine 28 after sending the address word sends 
ngl^her words, rather it waits for return data. Data is 
re|ft|ged because master state machine 37 in NIC 56 
rcac&to the arrival of a read address by requesting a 
r^^of : memory 60 via I/O bus 59 and corresponding 
'lc^[|bus bridge 12. This data is returned as if it were 
w^l^ata flowing from N IC 56 to NIC 1 1 , but without an 
'-"'"^address. Protocol decoder 33 reacts to this ad- 
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dressless data by routing it to read return FIFO 36, 
whereupon state machine 28 is released from its wait 
and the microprocessor 5's read cycle is allowed to com- 
plete. Should the address region be marked in NIC 56's 
bridge 12 as read-prefetchable, then a number of words 
are returned; if state machine 28 continues requesting 
data as if from a local bus burst read, then subsequent 
words are fulfilled directly from read return FIFO 36. 
[0049] Should NIC 56 need to raise an interrupt on 
microprocessors, remote interrupt generator 54 causes 
state machine 28 to send a word from NIC 56 to a mail- 
box register in NIC 1Vs bridge 12. This will have been 
configured by software to raise an interrupt on micro- 
processor 5. 

[0050] Inevitably, since the clocks 25 in NICs 11 and . 
56 will run at slightly different frequencies, there will be 
occasional overrun conditions. Where the command 
FIFO 34 exceeds a pre-programmed threshold value, 
an Xoff bit is sent to the corresponding protocol encoder 
29. This bit causes the encoder to request that the send- 
ing state machine 28 stops, if necessary in mid burst. 
Logic in bridge 1 2 takes care of restarting the data burst 
when the corresponding Xon is received some time lat- 
er. This logic calculates a new reference address for the 
unsent part of the data burst, using the reference ad- 
dress in the header of the whole data burst, and from a 
count of the number of data words which are sent before 
the transfer is stopped. As, in this embodiment, succes- 
sive data words in a burst have successively increment- 
ing destination addresses, the destination address of 
the first data word in the unsent part of the data burst 
can easily be calculated. 

[0051 ] It is also possible that data may be read out of 
FIFO 34 faster than it is written in. In the event of this 
happening, master state machine 37 uses pipeline de- 
lay 38 to anticipate the draining of FIFO 34 and to ter- 
minate the data burst on bcal bus 55. It then uses the 
CAM address latch/counter 41 to restart the burst when 
more data arrives in FIFO 34. 

[0052] Tripwires' are triggering values, such as ad- 
dresses, address ranges or other data, that are pro- 
grammed into the NIC to be matched. Preferably, the 
trigging values used as tripwires are addresses. To meet 
timing requirements during address match cycles (as 
data flows through the NIC), three CAM devices are 
pipelined to reduce the match cycle time from around 
70 nanoseconds to less than 30 nanoseconds. 
[0053] The programming of Tripwires takes place by 
microprocessor 5 writing to PCI bridge 12 via system 
controllers and I/O bus 9. For the purpose of writing the 
Tripwire data, CAM array 18, 19, 20 appears like con- 
ventional RAM to microprocessor 5. For write cycles, 
this is done by CAM controller 43 generating suitable 
control signals to enable all three CAMs 18, 19, 20 for 
write access. Address latch 44 passes data to the CAMs 
unmodified. Address multiplexer 41 is arranged to pass 
local bus data out on the CAM address bus where it is 
latched at the moment addresses are valid on the local 
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bujj^j£|atch 17. For read cycles, the process is similar, 
ejccept;that only CAM 18 is arranged to be enabled for 
read^ccess, and address latch/counter 44 has its data 
* flojlglirection reversed. So far as microprocessor 5 is 
5 concern ed, it sees the expected data returned, since the 
.memory arrays in CAMs 18, 19, 20 either contain the 
sarpe|clata, or internal flags indicating that particular 
segrnents of the memory array have not yet been written 
ari3 -should not participate in match cycles. 
10 [QP&4] Owing to the nature of the address/data bus 
bein^" 'comprised of bursts of data, according to the pre- 
ferred local protocol, the actual data stream cannot be 
used for monitoring address changes. A burst starts with 
thejaddress of the first data word followed by an arbitrary 
*s ny^6er ( of data words. The address of the data words 
'^ r — Elicit and increments from the start address. For 
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ndftnjal inbound or outbound data transfer operations, 



address latch/counter 44 is loaded with the address of 
e|clgnew data burst, and incremented each time a valid 
dajei|ijem is presented on internal local bus 55. CAM 
cp;n||6;i state machine 43 is arranged to enable each 
C^p,^|8, 19, 20 in sequence for a compare operation 
ia|^lch new address is output by latch/counter 44. This 
SQgy^ntial enabling of the CAMs combined with their 
l^tc^n^ properties permits the access time for a com- 
pir.jsp'n operation to be reduced by a factor of three 
(t|^|£belng three CAMs in this implementation, other 
Implementations being possible) from 70ns to less than 
30ns|ifhe CAM op -code for each comparison operation 
is^iitputfrom one of the internal registers 49 via address 
ir^ljiplexers 41 and 17. The op-code is actually latched 
by^dSress multiplexer 17 at the end of a read/write cy- 
ci^fjceeing the CAM address bus to return the index of 
irjia|9ned Tripwires after comparison operations. 
[0^5] The Tripwire data (i.e. the addresses to be 
mp^JoYed) is written to sequential addresses in the 
CAMfarray. During the comparison operation (cycle), all 



yaJ.alTnpwires are compared in parallel with the address 
pf,/^jcun'ent data, be it inbound or outbound. During 
th^peration, masking operations may be performed, 
dep^iVdlng on the type of CAM used, allowing certain 
b|s|^the address to be ignored during the comparison. 
InltHfeway, a Tripwire may actually represent a range of 
agcjresses rather than one particular address. 
[0JQi5£} : ' When the CAM array signals a match found (i. 
e|a^ripwire has been hit), it returns the address of the 
^n|wire (its offset in the CAM array) via the CAM ad- 
dress 15 us to the tripwire FIFO 42. Two courses of action 
are^ien possible, depending on how internal registers 
4S^§ye been programmed. 

t^^||j ( One course of action is for state machine 45 to 
request that an interrupt be generated by management 
!9glg§3. In this case, an interrupt is received by micro- 
processor 5, 



and software is run which services the in- 
te|^}oJ|: Normally this would involve microprocessor 5 
reaping the Tripwire address from FIFO 42, matching 
tlf^Sdress with a device-driver table, signalling the ap- 
pro^kte process, marking it runnable and reschedul- 



10/21/05, EAST 




2.0.1 .4 



11 



EP 1 302 853:A2; 




12 



ing. 

[0058] An alternative course of action is for state ma- 
chine 45 to cause records to be read from SRAM 23 
using state machine 46. A record comprises a number 
of data words; an address and two data words. These 5 
words are programmed by the software just before the 
Tripwire information is stored in the CAM. When a Trip- 
wire match is made, the address in LATCH 44 is left 
shifted by two to form an address index for SRAM 23. 
The first word is then read by state machine 46 and 10 
placed on local bus 55 as an address in memory 6. A 
fetch-and-increment operation is then performed by 
state machine 45, using the second and third words of 
the SRAM record to first AND and then OR, or else IN- 
CREMENT the data referred to in memory 6. A bit in the 15 
first word read by the state machine will indicate which 
operation it should take. In the case of an INCREMENT, 
the first data word also indicates the amount to incre- 
ment by. 

[0059] These alternatives enable the implementation 20 
of such primitives as an event counter incremented on 
tripwire matches, or the setting of a system reschedule 
flag. This mechanism enables multiple applications to 
process data without the requirement for hardware in- 
terrupts to be generated after receipt of each network 25 
packet. 

[0060] While in the case of the interrupt followed by a 
Tripwire FIFO read, the device driver is presented with 
a list of endpoints which require attention. This list im- 
proves system performance as the device driver is not 30 
required to scan a large number of memory locations 
looking for such endpoints. 

[0061 ] Since the device driver is not required to know 
where the memory locations which have been used for 
synchronisation are. It is also not required to have any 35 
knowledge or take part in the application level commu- 
nication protocol. All communication protocol process- 
ing can be performed by the application and different 
applications are free to use differing protocols for their 
own purposes, and one device driver instance may sup- 40 
port a number of such applications. 
[0062] There is also a problem connected with pro- 
gramming a DMA engine that is addressed by an aspect 
of the invention. Conventional access to DMA engines 
is moderated either by a single system device driver, 45 
which requires (slow) context switches to access, or by 
virtualisation of the registers by system page fault, also 
requiring (multiple) context switches. The problem is 
that it is not safe for a user level application to directly 
modify the DMA engine registers or a linked list DMA 50 
queue, because this must be done atomically. In most 
systems, user applications cannot atomically update the 
DMA queue as they can be descheduled at any mo- 
ment. 

[0063] The invention addresses this problem by using 55 
hardware FIFO 50 to queue DMA requests from appli- 
cations. Each application wanting to request DM A trans- 
fers sets up a descriptor, containing the start address 
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arid jfce length of the data to be transferred, in its local 
memory and posts the address of the descriptor to the 
"DM^gueue, whose address is common to all applica- 
tio^sjjhis can be arranged by mapping a single page 
containing the physical address of the DM A queue as a 
write^pnly page into the address space of all user appli- 
c^tlons as they are initialised. 

[00£4] As soon as DMA work queue FIFO 50 is not 
ert^ty,' local bus 55 is not busy and the DMA engine in 
bndge*;i2 is also not busy, Master/Target/DMA arbiter 
40 grants DMA state machine 51 access to local bus 55. 
U^g^the address posted by the application in FIFO 50, 
staj^i|nachine 51 then uses bridge 12 to read the de- 
sc|jgtor in memory 6 into the descriptor block 52. State 
ma|£jjie 51 then posts the start address and length in- 
|c^^^on held in block 52 into the DMA engine in bridge 

[0Q65]' When the DMA process is complete, bridge 12 
notifies" state machine 51 of the completion. The state 



rnacjjiriethen uses data from descriptor block 52 to write 
b'^c^a completion descriptor in memory 6. Optionally, 
anjigterrupt can also be raised on microprocessor 5, al- 
thojjjgh a Tripwire may already have been crossed to 
prj>|rjdd this notification early in order to minimise the 
dpiay.bringing the relevant application back onto micro- 
processor 5's run queue. This is shown later in this doc- 
umejit. 

[0066] Should queue 50 be full, then state machine 
5 1> writes a failure code back into the completion field of 
toe'qescriptor that the application has just attempted to 
placf jpn the queue. Thus the application does not need 
tbVreSd the status of the NIC in order to safely post a 
DMAVrequest. All applications can safely share the same 
h^td^are posting address, and no time-consuming vir- 
* "^"ation orsystem device driver process is necessary. 
Should any operation take longer than a preset 
fer of PCI cycles, timeout logic 61 is activated to 
Xe^^iaXe the current cycle and return an interrupt 
th^g^h block 53. 

[00681, Another aspect of the invention relates to the 
prc|ocql which is preferably used by the NIC. This pro- 
to^'plju'ses an address and some additional bits In its 
h||aJiBr. This allows the transfer of variable length pack- 
e^j^th simple routines for Segmentation and Reas- 
s|(ribly (SAR) that are transparent to the sending or re- 
codes. This is also done without the need to 
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hav;e;an entire packet arrive before segmentation, reas- 
sembly or forwarding can occur, allowing the data to be 
put-jut on the ongoing link immediately. This enables 
d^tp traverse many links without significantly adding 
to^fie} overall latency. The packets may be fragmented 
anj&palesced on each link, for example between the 
N|^nd a host I/O bus bridge, or between the NIC and 
ano^hW NIC. We term this cut-through routing and for- 
wjar^lhg. In a network carrying a large number of 
sj^arns, cut-through forwarding and routing enables 
sr^|l^)ackets to pass through the network without any 
delays caused by large packets of other streams. While 
ML 
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other network physical layers such as ATM also provide 
the ability to perform cut-through forwarding and routing, 
they do so at the cost of requiring all packets to be of a 
fixed small size. 

[0069] Figure 8 shows an example of how this proto- s 
col has been implemented using the 23-bit data transfer 
capability of HP's GLINK chipset (serialiser 14 and de- 
serialiser 16). PCI to local bus bridge 12 provides a bus 
of 32 address/data bits, 4 parity bits and 4 byte-enable 
bits. It also provides an address valid signal (AOS) which '0 
signifies that a burst is beginning, and that the address 
is present on the address/data bus. The burst continues 
until a burst last signal (BLAST) is set active, signifying 
the end of a burst. It provides a read/write signal, and 
some other control signals that need not be transferred 15 
to a remote computer. Figure BA shows how this proto- 
col is used to transfer an n data word burst 63. The data 
traffic closely mirrors that used on the PCI bus, but uses 
fewer signals. 

[0070] The destination address always precedes 20 
each data burst. Therefore, the bursts can be of variable 
size, can be split or coalesced, by generating fresh ad- 
dress words, or by removing address words where ap- 
plicable, in the preferred embodiment, sequential data 
words are destined for sequentially incrementing ad- 25 
dresses. However, data words having sequentially dec- 
rementing addresses might also be used, or any other 
pattern of addresses may be used so long as it remains 
easy to calculate. So far as the endpoints are corf- 
cerned, exactly the same data is transferred to exactly 30 
the same locations. The benefits are that packets can 
be of any size at all, reducing the overhead of sending 
an address; packets can be split (and addresses regen- 
eratedto continue) by network switches to provide qual- 
ity of service, and receivers need not wait for a complete 35 
packet to arrive to begin decoding work. 
[0071] Also, the destination address given in the 
header may be for the 'nth' data word in the burst, rather 
than for the first, although using the first data word ad- 
dress is preferred. 40 
[0072] Figure 8b shows how the protocol of Figure 8a 
is transcribed onto the G-LINK physical layer. The first 
word in any packet contains an 1 8-bit network address. 
Each word of 63 is split into two words in 64; the lower 
16 bits carry high and low addresses or data, corre- 45 
sponding to the address/data bus; the next 4 bits carry 
either byte enables or parity data. During the address 
phase, the byte enable field (only 2 bits of which are 
available, owing to the limitations of G-LINK) is used to 
carry a 2-bit code indicating read, write or escape packet so 
use. Escape packets are normally used to carry diag- 
nostic or error information between nodes, or as a 
means of carrying the Xon/X off-style protocol when no 
other data is in transit. The G-LINK nCAV signal corre- 
sponds to the ADS signal of 63; nDAV is active through- 55 
out the rest of the burst and the combination of nDAV 
inactive and nCAV inactive signals the end of a burst, 
or nCAV active indicates the immediate beginning of an- 



othe|burst 

PQJ3] Figure 8c, shows a read data burst 65; this is 
trjg^ame as a write burst 64, except data bit 16 is set 
toJphOn the outbound request, the data field contains 
th^etwork address for the read data to be returned to. 
Whjip the data for a read returns 66, it travels like a write 
buj^| : :.but is signified by there only being one nCAV ac- 
tiv^^ignifying the network address) along with the first 
wlp/dijAn additional bit, denoted FLAG in Figure 8, is 
useato carry Xon/Xoff style information when a burst is 
injpipgress. It is not necessary therefore to break up a 
bur^jh order to send an Escape packet containing the 
Xdn/Xbff information. The FLAG bit also serves as an 
additional end of packet indicator. 
[PQ^]- In Figure 8c, 67,68 shows an escape packet; 
aneMhe network address, this travels with 68 or without 
6^^ayload as defined by data bit 1 6 in the first word 
ofithe r burst. 

[OOJJ!]/ In a full networked implementation, an extra 
nej^grk address word may precede each of these pack- 
ets$£)ther physical layer or network layer solutions are 
po^ble, without compromise to this patent application, 
incj^iing fibre channel parts (using 8B/10B encoding) 
arto^pnventional networks such as ATM or even Ether- 
n|^The physical layer only needs to provide some 
means of identifying data from non-data and the start of 
one'burst from the end of a previous one. 
[Q076] A further aspect of the invention relates to the 
distribution of hardware around a network. One use of 
awetwork is to enable one computer to access a hard- 
waigi device whose location is physically distant. As an 
>le, consider the situation shown in Figure 9, 



w|jej:e ; it is required to display the images viewed by the 
cqirr^ra 70, (connected a frame-grabber card 69) on the 



nio.ntf6rwhich is, in turn, connected to computer 72. The 
N^||j3 is programmed from Boot ROM 22 to present 
the^jame hardware interface as that of the frame-grab- 
bp^ga^d 69. Computer 72 can be running the standard 
. appjipation program as provided by a third party vendor 
wfiichjs unaware that system has been distributed over 
a^netvyork. All control reads and writes to the frame- 
gr^^er 69, are transparently forwarded by the N IC 73, 
;'an'^Aere Is no requirement for an extra process to be 
pi^e|d;in the data path to interface between the appli- 
. csLt^air^ru n n i n g on CPU 74 and the NIC 73. Passive PCI 
l/g^Sack-plane 71 , requires simply a PCI bus clock and 
aroifer i.e., no processor, memory or cache. These func- 
tionsjcan be implemented at very low cost. 
[00777]. The I/O buses are conformant b PCI Local Bus 
Specification 2.1. This PCI standard supports the con- 
cept^ a bridge between two PCI buses. It is possible 
to^robram the NIC 73 to present the same hardware 
iri^|face as a PCI bridge between Computer 72 and 
pas&Ye back-plane 71 . Such programming would ena- 
bp^pjurality of hardware devices to be connected to 
bacj^pjane 71 and controlled by computer 72 without 
trfe^fequirement for 1 additional interfacing software. 
ApaM^' 1 should be clear that the invention will support 
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both general networking activity and this remote hard- 
ware communication, simultaneously using a single net- 
work card. 

[0078] A circular buffer abstraction will now be dis- 
cussed as an example of the use of the NIC by an ap- s 
plication. The circular buffer abstraction is designed for 
applications which require a producer/consumer soft- 
ware stream abstraction, with the properties of low la- 
tency and high bandwidth data transmission. It also has 
the properties of responsive flow control and low buffer 10 
space requirements. Fig. 10 shows a system comprising 
two software processes, applications 102 and 103, on 
different computers 100, 101 . Application 102 is produc- 
ing some data. Application 103 is awaiting the produc- 
tion of data and then consuming it. The circular buffer '5 
107, is composed of a region of memory on Computer 
101 which holds the data and two memory locations - 
RDP 1 06 and WRP 1 09. WRP 1 09 contains the pointer 
to the next byte of data to be written into the buffer, while 
RDP 1 06 contains the pointer to the last byte of data to 20 
be read from the buffer. When the circular buffer is emp- 
ty, then WRP is equal to RDP + 1 modulo wrap-around 
of the buffer. Similarly, the buffer is full when WRP is 
equal to RDP - 1 . There are also private values of WRP 

108 and RDP 111 in the caches of computer 100 and 25 
computer 101 respectively. Each computer 100,101 
may use the value of WRP and RDP held in its own local 
cache memory to compute how much data can be writ- 
ten to or read from the buffer at any point in time, without 
the requirement for communication over the network. 30 cj 
[0079] When the circular buffer 107 is created, the 
producer sets up a Tripwire 110, which will match on a 
write to the RDP pointer 1 06, and the consumer sets up 
a Tripwire 113, which will match on a write to the WRP 
pointer 109. 35 
[0080] If consumer application 103 attempts to read 
data from the circular buffer 1 07, it first checks to see if 
the circular buffer is empty. If so, application 103 must 
wait until the buffer is not empty, determined when WRP 

1 09 has been seen to be incremented. During this wait- to 
ing period, application 103 may either block, requesting 
an operating system reschedule, or poll the WRP 109 
pointer. 

[0081] If producer application 1 02 decides to write to 
the circular buffer 107, it may do so while the buffer is 45 
not full. After writing some data, application 1 02 updates 
its local cached value of WRP 108, and writes the up- 
dated value to the memory location 109, in computer 
1 01 . When the value of WRP 1 09, is updated, the Trip- 
wire 113, will match as has been previously described, so 
[0082] If consumer application 103 is not running on 
CPU 118 when some data is written into the buffer and 
Tripwire 113 matches, NIC 115 will raise a hardware in- 
terrupt 1 1 4. This interrupt causes CPU 1 1 8 to run device 
driver software contained within operating system 118. 55 
The device driver will service the interrupt by reading 
the tripwire FIFO 42 on NIC 1 1 5 and determine from the 
value read, the system identifierfor application 1 03. The 
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iVjc;e k driver can then request that operating system 
^reschedule application 103. The device driver 



wo^plhen indicate that the tripwire 1 1 3 should not gen- 
erate^ hardware interrupt until application 1 03 has been 
htjx^descheduled and subsequently another Tripwire 
maljcfthas occurred. 

[0083]; Note that the system identifierfor each running 
.appljcation is loaded into internal registers 49, each time 
tr^^perating system reschedules. This enables the NIC 
to^.|etermine the currently running application, and so 
— '"S^the decision whether or not to raise a hardware 



intejjjupt for a particular application given a Tripwire 
pOjg}** Hence, once consumer application 103 is 



ao^aln^fiinning on the processor further writes to the cir- 
c^j- : buffer 1 07, by application 102, may occur without 
tW^ering further hardware interrupts. Application 103 
n#r^ads data from the circular buffer 107. It can read 
daKuntil the buffer becomes empty (detected by com- 
panng the values of RDP and WRP 1 1 1 , 1 09). After read- 
in^^pplication 102 will update its local value of RDP 
1 J^nd finally writes the updated value of RDP to mem- 
or^Scation 106 over the network. 
[0085]i| If producer application 102 had been blocked 
^ijl buffer, this update of RDP 1 06 would generate 
ijivire match 110, resulting in application 102, being 
icked and able to write more data into the buffer 



[0086],. in normal operation, application 1 02 and appli- 
c|t|pi|103 could be operating on different parts of the 
cjr^ujan buffer simultaneously without the need for mu- 
ti^ij'exclusion mechanisms or Tripwire. 
[(^387p^The most important properties of the data 
|^ct|fe are that the producer and the consumer are 
a|||gojprocess data without hindrance from each other 
ar|]^]||t flow control is explicit within the software ab- 
str^cjion. Data is streamed through the system. The 
consumer can remove data from the buffer at the same 
time^as the producer is adding more data. There is no 
dapger/of buffer over-run, since a producer will never 
tra^sWrt more data than can fit in the buffer. 
[008^' The producer only ever increments WRP 10B, 
J |||tnd reads RDP 106, and the consumer only ever 



1 _ 

iripriiments RDP 106, 111, and reads WRP 109. Incon- 
sjs^ricies in the values of WRP and RDP seen by either 
tr||^rpducer or consumer either cause the consumer to 
notj^rocess some valid data (when RDP 106 is incon- 
sislent with 111), or the producer to not write some more 
data!(w t hen WRP 109 is inconsistent with 108), until the 
ini3ofJ|istency has been resolved. Neither of these oc- 
pyrjsnces cause incorrect operation or performance 



degradation so long as they are transient. 
[008?| It should also be noted that on most computer 
arg^ectures, including the Alpha AXP and Intel Pen- 
tium^nges, computer 100 can store the value of the 
^JPpp pointer in its processor cache, since the pro- 
d|^||application 102 only reads the pointer 106. Any 
~"T]}pte writes to the memory location of the RDP pointer 
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106 will automatically Invalidate the copy in the cache 
causing the new value to be fetched from memory. This 
process is automatically carried out and managed by the 
system controller 8. In addition, since computer 101 
keeps a private copy of the RDP pointer 111 in its own 5 
cache, there is no need for any remote reads of RDP 
pointer values during operation of the circular buffer. 
Similar observations can also be made for the WRP 
pointer 109 in the memory of computer 101 and the 
WRP pointer 108 in the cache of computer 100. This 10 
feature of the buffer abstraction ensures that high per- 
formance and low latency are maintained. Responsive 
application level flow-control is possible because the 
cached pointer values can be exposed to the user-level 
applications 102, 103. 

[0090] A further enhancement to the above arrange- 
ment can be used to provide support for applications 
which would like to exchange data in discrete units. As 
shown in Fig. 11 , and in addition to the system described 
in Fig. 10. The system maintains a second circular buffer 
1 27, of updated WRP 1 29 values corresponding to buff- 
er 125. This second buffer 127 is used to indicate to a 
consumer how much data to consume in order that data 
be consumed in the same discrete units as it were pro- 
duced. Note that circular buffer 1 25 contains the data to 
be exchanged between the applications 122 and 123. 
[0091] The producer, application 122 writes data into 
buffer 1 25, updating the pointer WRP 1 29, as previously 
described. Once data has been placed in buffer 125, ap- 
plication 1 22 then writes the new value of the WRP 1 29 
pointer into buffer 127. At the same time it also manip- 
ulates the pointer WRP 1 31 . If either of these write op- 
erations does not complete then the application level 
write operation is blocked until some data is read by the 
consumer application 123. The Tripwire mechanism can 35 
be used as previously described, for either application 
to block on either a full or empty buffer pair. 
[0092] The consumer application 123 is able to read 
from both buffers 125 and 127, in the process updating 
the RDP pointers 133, 135 in its local cache and RDP *o 
pointers 124, 126 over the network in the manner pre- 
viously described. A data value read from buffer 1 27 in- 
dicates an amount of data, which had been written into 
buffer 125. This value may be used by application level 
or library software 1 23, to consume data from buffer 1 25 45 
in the same order and by the same discrete amounts as 
it were produced by application 122. 
[0093] The NIC can also be used to directly support 
a low latency Request/Response style of communica- 
tion, as seen in client/server environments such as 50 
Common Object Request Broker Architecture (CORBA) 
and Network File System (NFS) as well as transactional 
systems such as databases. Such an arrangement is 
shown in Fig. 12, where application 142 on computer 
1 40 acts as a client requesting service from application 55 
143 on computer 141, which acts as a server. The ap- 
plications interact via memory mappings using two cir- 
cular buffers 144 and 145, one contained in the main 



merhdfy of each computer. The circular buffers operate 
■ asfpreyiously described, and also can be configured to 
transfer data in discrete units as previously described. 
Application 142, the client, writes a request 



Indirectly into the circular buffer 145, via the memory 
mapped connection(s), and waits for a reply by waiting 
onLdata to arrive in circular buffer 144. Most Request/ 
Response systems use a process known as marshalling 
tO;CfOhstruct the request and use an intermediate buffer 
| jn;memory of the client application to do the marshalling. 
j'jljKewise marshalling is used to construct a response, 
jwifean intermediate buffer being required in the mem- 
fi»ryfo£the server application. Using the present inven- 
j|gn^rjiarsh ailing can take place directly into the circular 
'5 i{kufferVi 45 of the server as shown. No intermediate stor- 
^fage^of^the request is necessary at either the client or 
.s^ryepcomputers 140, 141. 

[Q0£5] The server application 1 43 notices the req uest 
*'f .(^s^iBly using the Tripwire mechanism) and is able to 
b^ip|unmarshalling the request as soon as it starts to 
ar^v^in the buffer 1 45. It is possible that the server may 
ha^Jj^tarted to process the request 149 while the client 
|s^ r |ti| marshalling and transmitting, thus reducing laten- 
cyjn;jtie communication. 
25 rb'096ll;.j After processing the request, the server writes 
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y^plfiy After processing the request, the server writes 
thei/eply 146 directly into buffer 144, unblocking appli- 
pj|i^i42 (using the Tripwire mechanism), which then 
un^rshalls and processes the reply 148. Again, there 
Isa^rieed for intermediate storage, and unmarshalling 
byrtfie< client may be overlapped with marshalling and 
transmission by the server. 

[0 f 09J] A further useful and novel property of a Re- 
qtijSsJ/Response system built using the present inven- 
tipnjls that data may be written into the buffer both from 
software running on a CPU, or any hardware device 
cpnjajned in the computer system. Fig. 15 shows a Re- 
quesH/Response system which is a file serving applica- 
tion^he client application 262 writes a request 267 for 



some^data held on disks controlled by 271 . The server 
application 263 reads 269 and decodes the request from 
ular buffer 265 in the manner previously de- 



scribed. It then performs authentication and authorisa- 
tion „pn the request according to the particular applica- 

ill 

[0Q98]; If the request for data is accepted, the server 
al^lication 263 uses a two-part approach to send its re- 



pt£||iptly, it writes, into the circular buffer 264, the soft- 
w^r^generated header part of the reply 266. The server 
a^pjjpation 263 then requests 273 that the disk control- 
lerkMsend the required data part of the reply 272 over 
ttyejrietwork to circular buffer 264. This request to the 
disi<|c6ntrollertakes the form of a DMA request, with the 
targ^address being an address on I/O bus 270 which 
ha| ; |e'en mapped onto circular buffer 264. Note that the 
correct offset is applied to the address such that reply 
data!>272 from the disk is placed immediately following 
iJ ^"- , "data 266. 

Before initiating the request 273, the server ap- 
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plication 263 can ensure that sufficient space is availa- 
ble in the buffer 264 to accept the reply data. Further, it 
is not necessary for the server application 263 to await 
the completion request 273. It is possible for the client 
application 262 to have set a Tripwire 274 to match once s 
the reply data 272 has been received into buffer 264. 
This match can be programmed to increment the WRP 
pointer associated with buffer 264, rather than requiring 
application 263 to increment the pointer as previously 
described. If a request fails, then the client application 10 
262 level timeout mechanism would detect and retry the 
operation. 

[01 00] It is also possible for the client application 262 
to arrange that reply data 272 be placed in some other 
data structure, (such as a kernel buffer-cache page), '5 
through manipulation of 1 69 and 1 67 as described later. 
This is useful when 264 is not the final destination of the 
rept data, so preventing a final memory copy operation 
by the client. Server application 263 would be unaware 
of this client side optimisation. 20 
[01 01] By use of this mechanism, the processing load 
on the server is reduced. The requirement for the server 
application to wait for completion of its disk requests is 
removed. The requirement for high bandwidth streams 
of reply data to pass through the server's system con- 25 
troller, memory, cache or CPU is also removed. 
[01 02] As previously stated, the NIC of the present In- 
vention could be used to support the Virtual Interface 
Architecture (VIA) Standard. Fig. 13 shows two applica- 
tions communicating using VIA. Application 152 sends 30 
data to application 153, by first writing the data to be 
sent into a region of its memory, shown as block 154. 
Application 152 then builds a transmit descriptor 156, 
which describes the location of block 154 and the action 
required by the NIC (in this case data transmission). 35 
This descriptor is then placed onto the TxQueue 158, 
which has been mapped into the user-level address- 
space of application 152. Application 152 then finally 
writes to the doorbell register 1 60 in the NIC 1 62 to notify 
the NIC that work has been placed on the TxQueue 158. *o 
[0103] Once the doorbell register 1 60 has been writ- 
ten, the NIC 162 can determine, from the value written, 
the address in physical memory of the activated Tx- 
Queue 158. The NIC 152 reads and removes the de- 
scriptor 156 from theTxQueue 158, determines from the 45 
descriptor 156, the address of data block 154 and in- 
vokes a DMA 1 64 engine to transmit the data contained 
in block 1 54. When the data is transmitted 1 68, the NIC 
162 places the descriptor 156 on a completion queue 
166, which is also mapped into the address space of 50 
application 1 52, and optionally generates a hardware in- 
terrupt. The application 152 can determine when data 
has been successfully sent by examining queue 166. 
[0104] When application 153 is to receive data, it 
builds a receive descriptor 157 describing where the in- 55 
coming data should be placed, In this case block 155. 
Application 153 then places descriptor 157 onto Rx* 
Queue 1 59, which is mapped into its user-level address- 
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ls||r^(j§1 to indicate that its RXQueue 159 has been ac- 
tivated. It may choose to either poll its completion queue 
.^ : §v/aiting for data to arrive, or block until data has 
an^£etf and a hardware interrupt generated. 
[Orj^j, The NIC 165 in computer 151 services the 
doipn^ell register 1 61 write by first removing the descrip- 
to]^7 from the RxQueue 159. The NIC 165 then lo- 
cates the physical pages of memory corresponding to 
block 155 and described by the receive descriptor 157. 
The VlA standard allows these physical pages to have 
beenjpreviously locked by application 1 53 (preventing 
t^ejyijrtual memory system moving or removing the pag- 
estffpm physical memory). However, the NIC is also ca- 
pabfijlof traversing the page-table structures held in 
pHysical memory and itself locking the pages. 
[f£l*0^j : The NIC 1 65 continues to service the doorbell 
register write and constructs a Translation Look-aside 
(T^B) ? entry 167 located in SRAM 23. When data arrives 
c^nre^sponding to a particular VIA endpoint, the incoming 
addrB^s matches an aperture 169 in the NIC, which has 
bf^rjnarked as requiring a TLB translation. This trans- 
latj^is carried out by state machine 46 and determines 
t[1||j|Bysical memory address of block 155. 
Ityffljf; The TLB translation, having been previously 
set'tipr occurs with little overhead and the data is written 

••.yiv'VPiVT 

t£5\jto, appropriate memory block 155. A Tripwire 171 
w^lj^ave been arranged (when the TLB 167 entry was 
constructed) to match when the address range corre- 
sponding to block 155 is written to. This Tripwire match 
ca^sejs the firmware 1 73 (implemented in state machine 
5jlSl9;place the receive descriptor 157 onto completion 



queue 163 to invalidate the TLB mapping 167 and op- 
tig^alry generate an interrupt. If the RxQueue 159 has 
bejjr^loaded with other receive descriptors, then the 
nex|a£scriptor is taken and loaded into the TLB as pre- 
v (o^|j!yj described. If application 153 is blocked waiting 
fp^ata to arrive, the interrupt generated will result, (af- 
te|||ievice driver has performed a search of all the com- 
pletign queues in the system), in application 153 being 
re^-|||h;eduled. If there is no TLB mapping for the VIA Ap- 
erturei addresses, or the mapping is Invalid, an error is 
raigpd.uslng an interrupt. If the NIC 1 65 is in the process 
bfef&lP^ding the TLB 167 when new data arrives, then 



h|r^dware flow control mechanism 31 is used to control 
th^e^ata until a path to the memory block in computer 
15iTifias been completed. 

As an optional extension to the VIA standard, 
thMNjC could also respond to Tripwire match 171 by 



placirig an index on Tripwire FIFO 42, which could ena- 
b|^||hi device driver to identify the active VIA endpoint 
wj^i'pyt searching all completion queues in the system. 
(q*109J* . This method can be extended to provide sup- 
P^|[oj: 120 and the forthcoming Next Generation I/O 
(^j^y standard. Here, the transmit, receive and com- 
ptgl^^queues are located on the NIC rather than in the 
p|ysi§al memory of the computer, as is currently the 
cas^fpr the VIA standard. 
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[0110] As mentioned previously, another aspect of 
this invention is its use in providing support for the out- 
bound streaming of data through the NIC. This setup is 
described in Fig. 14. !t shows a Direct Memory Access 
(DMA) engine 1 82 on the NIC 1 83, which has been pro- s 
grammed in the manner previously described by a 
number of user-level applications 184. These applica- 
tions have requested that the NIC 183 transfer their re- 
spective data blocks 181 through the NIC 1B3, local bus 
1 89, fibre-optic transceiver 190 and onto network 200. 10 
After each application has placed its data transfer re- 
quest onto the DMA request queue 1 85, it blocks, await- 
ing a re-schedule, initiated by device driver 187. It can 
be important that the system maintains fair access be- 
tween a large number of such applications, especially 15 
under circumstances where an application requires a 
strict periodic access to the queue, such as an applica- 
tion generating a video stream. 
[0111] Data transferred overthe network by the DMA 
engine 182, traverses local bus 189, and is monitored 20 
by the Tripwire unit 186. This takes place in the same 
manner as for received data, (both transmitted and re- 
ceived data pass through the NIC using the same local 
bus 55). 

[0112] Each application, when programming the DMA 25 
engine 182 to transmit a data block, also constructs a 
Tripwire which is set to match on an address in the data 
block. The address to match could indicate that all or a 
certain portion of the data has been transmitted. When 
this Tripwire fires and causes a hardware interrupt 1 88, 30 
the device driver 187 can quickly determine which ap- 
plication should be made runnable. By causing a system 
reschedule, the application can be run on the CPU at 
the appropriate moment to generate more DMA re- 
quests. Because the device driver can execute at the 35 
same time that the DMA engine is transferring data, this 
decision can be made in parallel to data transfer oper- 
ations. Hence, by the time that a particular application's 
data transfer requests have been satisfied, the system 
can ensure that the application be running on the CPU 40 
and able to generate more requests. 
[0113] Figure 16 illustrates a generalised apparatus 
or arrangement for synchronising an end-point applica- 
tion using a tripwire. An end-point is a final destination 
for an information stream and is the point at which 45 
processing of the information takes place. Examples of 
end-points include a web, a file, a database server and 
hardware devices such as a disk or graphics controller. 
An end-point may be running an operating system and 
a number of data processing applications and these are 50 
referred to as end-point applications. Thus, examples of 
end-point applications include an operating system or a 
component thereof, a network protocol stack, and any 
application-level processing. Arrangements such as 
network switches and routers do not constitute end- 55 
points or end-point applications because their purpose 
is to ensure that the Information is delivered elsewhere. 
[0114] The arrangement comprises a computer 300 



which Is optionally connected to other computers 301 
antf tf 302 via a network 303. The computer 300 compris- 
es^a prog ram memory (illustrated by way of example on- 
ly^s'^aYead only memory (ROM) 305) which contains a 
pr^pram for controlling the computer to synchronise the 
eno^point application in accordance with an address- 
basecj.event in an information stream on an information 
pathv/ay 307, such as a bus, within the computer. The 
inf^rjmation stream may be wholly within the computer, 
fdr^example from another application performed by the 
computer 300, or may be from a remote source, such 
ashram the network 303. 

P3^||^ The bus 307 is connected to a memory 308 in 
the^d-polnt application 306, which also comprises a 
. cpj^generator 309 and an action generator 310. The 
cpoj^generator 309 supplies codes to a comparator 
wtyph': is illustrated as a content addressable memory 
(CAM) 311. The CAM 311 has another input connected 
to^tfie bus 307 and is arranged to perform a comparison 
between each entry in the CAM and the information 
s^eam on the bus 307. When a match is found, the CAM 
sem&;a signal to the action generator 310 which per- 
fo'rh^an action which is associated with an address- 
bas^d event in the information stream. 

In atypical example of use of the synchronising 



arn|ngement, the end-point application 306 sets a trip- 
w^ltibr example to be triggered when data relating to 
anlend-point address or range of end-point addresses 
In^^memory 308 are present on the bus 307. The code 
gjtrjerptor 309 supplies a code which is written into the 
C§M:j311 and which comprises the destination memory 
address of the data or possibly part of this address, such 
a|f|He.most significant bits when a range of addresses 



isjtp^e monitored. It is also possible to enter a code 
w^jchjrepresents not only the address or range of ad- 
dresses but also part or all of one or more items of data 
v^icjr|ire expected in the information stream. The CAM 
3|^|cpmpares the address of each data burst on the bus 
3oJ|and possibly also at least some of the data of each 
byrs|if,with each code stored in the CAM 311 and sup- 
plied, signal to the action generator 31 0 when a match 
is^fggnd. The action generator 31 0 then causes the ap- 
pjp"f?$ate action to be taken within the end-point appll- 
catigji 306. This may be a single action, several actions, 
or^ijf ' or more specific actions which are determined 
npVoniy by the triggering of the tripwire but also by the 
dat^jirithin the information stream, for example arriving 
af thei' appropriate location or locations in the memory 

[04jj^l| As mentioned herein before, the information 
stream 307 may be wholly internal to the computer 300 
aplpLft .example of this is an application-to-application 
sfjgajji of information where both applications are run- 
rt|n^for example alternately, on the computer 300. 
wer, the information stream may be partly or wholly 



frc^outside the computer 300, as illustrated bythebro 
•ker|jjne connection from the bus 307 to the network 303 
Withe information stream may be from a switch fab 
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ric, a network, or a plurality of sources. A switch fabric 
is a device which has a plurality of inputs and outputs 
and which is capable of forwarding data from each input 
to the appropriate output according to routing informa- 
tion contained within the data. A switch fabric may alter- 5 
natively be wholly contained within the computer. The 
information stream preferably has a data burst arrange- 
ment as described hereinafter and, in the case of a plu- 
rality of sources, the data bursts may arrive from any of 
the sources at any time, which amounts to multiplexing, w 
[0118] Figure 17 shows an arrangement which illus- 
trates two possible modifications to the arrangement 
shown in Figure 1 6. In this case, the bus 307 is connect- 
ed to an input/output bus 312 of the end-point applica- 
tion 306 within the computer 300. This represents an '5 
example of a hardware end-point for the information 
stream but other types of hardware end-points are pos- 
sible, such as active controllers, and may be located 
"outside" the application 306. An example of an active 
controller is a disk controller. 20 
[0119] The arrangement shown in Figure 17 also dif- 
fers from that shown in Figure 1 6 in that the tripwire may 
be triggered by an address-based event in the informa- 
tion stream on the bus 307 which does not exactly match 
any of the codes stored in the CAM 311 . Instead, the 25 
information from the information stream on the bus 307 
first passes through a process 313 before being sup- 
plied to the CAM for comparison with each of the stored 
codes. 

[0120] One application of this is for the case where 30 
the information stream comprises packets or bursts of 
data starting with an address, for example correspond- 
ing to an address in the memory 308 to which the first 
item of data after the address in the packet or burst is 
allocated. Subsequent items of data are to be allocated 35 
to consecutive addresses, for example such that each 
item of data in the burst is to be allocated to the next 
highest address location after the preceding data item. 
Thus, the address at the start of each burst relates to 
the first data item and the following data item addresses 40 
can be inferred by incrementing the address upon the 
arrival of the second and each subsequent item of data. 
[0121] The application 306 can cause the code gen- 
erator 309 to store in the CAM 311 a code which corre- 
sponds to an implied address in the actual information 45 
stream appearing on the bus 307. The process 313 de- 
tects the address at the start of each data burst and sup- 
plies this to the CAM 311 with the arrival of the first data 
item. As each subsequent data item of the same burst 
arrives, the process 313 increments the address and 50 
supplies this to the CAM 311. This allows a tripwire to 
be triggered when, for example a data item having an 
implied address is present on the bus 307 because the 
CAM can match the corresponding stored code with the 
address supplied by the process 31 3. 55 
[0122] As mentioned hereinbefore, the action gener- 
ator 31 0 can cause any one or more of various different 
actions to be triggered by the tripwire. The resulting ac- 
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tion.may be determined by which tripwire has been trig- 
gered i.e. which code stored in the CAM 31 1 has been 
rnajctied. It is also possible for the action to be at least 
par^y'determined by the data item which effectively trig- 
ggpgj'the tripwire. Any action may be targetted at the 
co|nDuter containing the tripwire or at a different com- 
put'er^, Various possible actions are described hereinaf- 
tetfastypical examples and may be performed singly or 
in^eJiy appropriate combination for the specific applica- 
tion^and may be targetted at the computer containing 
t^tppwire or at a different computer. 
jPjlj?3] Figure 1 8 illustrates the action generator 310 
raising; an interrupt request IRQ and supplying this to 
tKeiJjjiterrupt line of a central processing unit (CPU) 320 
o(thj|computer300. Figure 1 9 illustrates the action gen- 
er^6'r.;310 setting a bit in a bitmap 321, for example in 
the^emory 308. These two actions may be used inde- 
pendently of each other or together. For example, the 
action generator may raise an interrupt request if an ap- 
plication which requires data corresponding to the trip- 
wjre>|3 not currently running but is runnable; for example 
itjhasj not exhausted its time-slice. Otherwise, for exam- 
pjb|jffthe application is awaiting rescheduling, the rele- 
y^nifbit in the bitmap 321 may be set. The operating 
sysjeni may periodically check the bitmap 321 for 
changes and, as a result of the arrival of the relevant 
d^tajfor an application which is presently not running, 
rrj^declde to reschedule or wake up the application. 
[0:j24]^ Figure 20 illustrates another type of action 
wm^jfnay be performed as a result of detection of the 
aqarjess-based event. In this example, a counter 322, 
fomexample whose count is stored within the memory 
3Q8^§jincremented in response to triggering of the trip- 
w^e^lhcrementing may take place as a result of any trip- 
w|re?b§ing triggered or only by one or more specific trip- 
w^res^depending on the specific application. 

Figure 21 illustrates another action which is 
lat, when the or the appropriate tripwire is trig- 
ggfgQ'a predetermined value "N" is written to a location 
"^J^jpwn at 323 as being in the memory 308 (or being 
mapped thereto). 

pjrjJiiS]} Figure 22 illustrates another combination of 
actios which may be used to indicate that an applica- 
tiqnfsjibuld be awakened or rescheduled. When a trip- 
wire|| triggered, an interrupt request is supplied to the 
CFJp320 and a "runnable bit" for a specific application 
is?|e|at location 324 in the memory 308. The operating 
SYs|ern of the computer 300 responds to the interrupt 
revest by waking up or rescheduling the application 
wHpse' runnable bit has been set. 



BUCj 



Figure 23 illustrates an action which modifies 
entries in the CAM 311 in response to triggering of a 
triple. Any form of modification is possible. For exam- 
plefijih'o code which triggers the tripwire may be deleted 
ff^^yrther tripwires are required forthe same address- 
tfg^e^event. As an alternative, the code may be modi- 
fied solas effectively to set a different but related tripwire, 
^m^fier possibility is to generate a completely new 
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code and supply this to the CAM 311 in order to set a 
new unrelated tripwire. 

[0128] Figure 24 illustrates the format of a data burst, 
a sequence of which forms the information stream on 
the bus 307. The data burst comprises a plurality of 5 
items which arrive one after the other in sequence on 
the bus. The first item is an address A(n) which is or 
corresponds to the end-point address, for example in 
the memory 308, for receiving the subsequent data 
items. This address is the actual address n of the first to 
data item D 1 of the burst, which immediately follows the 
address A(n). The subsequent data items D 2 , D 3 ... t D p 
arrive in sequence and their destination addresses are 
implied by their position within the burst relative to the 
first data item D1 and its address n. Thus, the second '5 
data item D 2 has an implied address n + 1 , the third data 
item D 3 has an implied address n + 2 and so on. Each 
data item is written or supplied to the implied address 
as its destination address. 

[0129] This data burst format may be used to frag- 20 
ment and coalesce bursts as the data stream passes 
through a forwarding unit 330, such as a network inter- 
face card or a switch, of an information pathway. For 
example, the forwarding unit can start to transmit a burst 
as soon as the first data item has arrived and does not 25 
have to wait until the whole data burst has arrived. 
[0130] Figure 25 illustrates an example of this In which 
an interruption in the data burst occurs. The forwarding 
unit 330 has already started transmission of the burst 
and the first r data items 331 together with the burst ad- 30 
dress have already been forwarded. The remainder 332 
of the burst has not yet arrived and the forwarding unit 
330 terminates forwarding or transmission of that burst. 
[0131] When the remainder 332 of the burst starts to 
arrive, the forwarding unit 330 recalculates the destina- 35 
tion address A(r +1 ) for the remainder of the burst and 
inserts this in front of the data item D r+1 . This is trans- 
mitted as a further burst 333 as illustrated in Figure 26. 
[0132] This technique may be used even when the 
whole burst is available for forwarding by the forwarding 40 
unit 330. For example, the forwarding unit 330 may ter- 
minate transmission of a particular burst before comple- 
tion of transmission for reasons of arbitration between 
a number of competing bursts or for flow control rea- 
sons. Thus, individual data bursts can be forwarded in 45 
tact or can be sent in two or more fragments as neces- 
sary or convenient and all such bursts are treated as 
valid bursts by any subsequent forwarding units. 
[0133] Figure 27 illustrates an alternative situation in 
which the forwarding unit has an internal buffer 335 so 
which contains first and second bursts 336 and 337. In 
this case, the implied address of the first data item D n + 
1 of the second burst 337 immediately follows the im- 
plied address of the last data item D n of the first burst 
336. The forwarding unit checks for such situations and, 55 
when they are found, coalesces the first and second 
bursts into a coalesced burst 338 as shown in the lower 
part of Figure 27. The forwarding unit then transmits a 
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siQgJ^contiguous burst, which saves the overhead of 
tl^^cess address information (which is deleted from 
trtelgecond burst). Any subsequent forwarding units 
th^n^treat the coalesced burst 338 as a single burst. 
Ppsij:' The format of the data burst allows such frag- 
~ tion or merging of bursts to take place. This in turn 
^forwarding units to transmit data as soon as it 
rj^slso as to reduce or minimise latency. Also, bursts 
p^arj^length or number of data items can be handled 
w_(jic||.jmproves the flexibility of transmission of data. 
[01,36] j Figure 28 illustrates an example of communi- 
^a|fenjbetween an application, whose address space is 
shown at 340, and remote hardware 341 via a network 
3£ Jsuch that the network 303 is "transparent" or "invis- 
ible^g ( each of the application and the remote hardware 
! 3^|?fhe address space 340 contains mapped configu- 
rati^fijdata and registers of the remote hardware as in- 
d^tecl at 342. This is mapped onto the system input/ 
» b^ut bus 343 to which a network interface card 344 is 
' competed. The network interface card 344 is loaded 
w^|[configu ration and register data corresponding to 
tr^pmote hardware 341. All application requests are 
.fp^warded over the network 303 transparently to the re- 
J ^Qjg^nardware 341 so that the remote hardware ap- 
.Dea^.as though it is local to the application and the net- 
work's^^ invisible. 

p)^36]-/The remote hardware 341 is connected to a 
■;pasj^e Input/output bus 345 which is provided with a 
• ne^prk interface card 346 for interfacing to the network 
30^Tj)e configuration and registers of the remote hard- 
w^re^e illustrated at 347 and are mapped ultimately to 
thjbjreflion 342 of the address space 340 of the applica- 
tion^gain, the network is invisible to the remote hard- 
wares^ and the remote application appears to be local 

[O^Tj; When the application sends a request to the 
re^aje hardware 341 , for example requesting that the 
i^jricS| hardware supply data to be used in or processed 
byii^application, this is written in the space 342 which 
isi'r|iapped to the system input/output bus 343. The net- 
wo|^lnterface card 344 sends read/write requests over 
trfg,^Uvork 303 to the card 346, which supplies these 
viSln,^ passive Input/output bus 345 to' the remote hard- 
warej-341 . Viewed from the remote hardware 341, the 
bus;345 appears equivalent to the bus 343. 
'iO/i$5UP The remote 1 hardware 341 may supply an in- 
ter^ugt and/or data for the application to the bus 345. 
Ag|ifl£jthe network interface card 346 sends this via the 
network 303 to the card 344. The network interface card 
3^Sdpplies an interrupt request to the computer run- 
nin^Uhe application and writes the data on behalf of the 
rernote hardware to the space 342 in the address space 
3^^the application. Thus, to the application, the re- 
mbt0; hardware 341 appears to be connected directly to 
tltei6us , 343. 

p)^j39]i - Although implementations of tripwires have 
be^described in detail hereinbefore with reference to 
thie^rifjwire unit 1 shown in Figure 29 associated with 
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the network interface card 350, tripwires may be imple- 
mented at other points in a system as illustrated by trip- 
wire units 2 to 5 in Figure 29. The system comprises a 
disk controller 351 connected to an input/output bus 
307b and the tripwire unit 2 is implemented as part of s 
the disk controller 351 . Such an arrangement allows trip- 
wire operations to inform applications of any character- 
istic data transfer to or from the disk controller 351 . Such 
an arrangement is particularly useful where the control- 
ler 351 is able to transfer data to and from a non-con- 10 
tiguous memory region corresponding to user-level buff- 
ers of an application. This allows data transfer and ap- 
plication level notification to be achieved without requir- 
ing hardware interrupts or kernel intervention. 
[01 40] The tripwire unit 3 is associated with a system *5 
controller 352 connected to a host bus 307a and the in- 
put/output bus 307b. Such an arrangement allows trip- 
wire operations to inform applications of any character- 
istic data transfer to or from any device in the computer 
system. This includes hardware devices, such as the 20 
disk controller 351 and the network interface card 350, 
and, in the case of a system employing several CPUs, 
enables an application running on one of the CPUs to 
synchronise on a data transfer to or from an application 
running on another of the CPUs. Similarly, atripwire may 25 
be used for synchronisation between applications run- 
ning on the same CPU. This reduces the need for other 
mechanisms such as spin locks where both applications 
are required to operate in lock-step with the data trans- 
fer. 30 
[0141] Tripwire units 4 and 5 are implemented in the 
CPU 320 or the memory 308. This is generally equiva- 
lent to the tripwire unit 3, where all data transfers in the 
system can be monitored. However, the tripwire unit 4 
may monitor data written by an application to cache, 3$ 
which may not appear on the host bus 307a. 
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1 . A method of synchronising between a sending ap- 
plication on a first computer and a receiving appli- 
cation on a second computer, each computer hav- 
ing a main memory, and at least one of the comput- 
ers having an asynchronous network interface, 45 
comprising the steps of: 

providing the asynchronous network interface 
with a set of rules for directing incoming data to 
memory locations in the main memory of the so 
second computer; 

storing in the network interface one ormore trig- 
gering value(s), each triggering value repre- 
senting a state of a data transfer between the 
applications; receiving, at the network inter- 55 
face, a data stream being transferred between 
the applications; 

comprising at least part of the data stream re- 



ti&f- ceived with the stored triggering values; 



Vt^V* if any compared part of the data stream match- 
; -es any triggering value, indicating that the trig- 
gering value has been matched; and 
storing the data received in the main memory 
of the second computer at one or more memory 
location(s) in accordance with the said rules. 



*.T&$method according to claim 1 , in which the step of 
-..^^'roviding the asynchronous network interface with 
( fe^set of rules comprises the step of establishing a 
* r %mapping between information contained within the 




pM^coming data stream and one or more memory lo- 
^Mcation(s) of the main memory of the second com- 
l#$puter. 

■'Mm 

3wA*method according to claim 2, in which the asyn- 
cl^jphfonous network interface is a memory mapped 
"Tfetwork interface, and in which the step of providing 
he memory mapped network interface with a set of 
Srqjes comprises the step of establishing a mapping 
between addresses contained within the incoming 

• iiNi? a stream an( * one or more memory location (s) of 
■ sEphe main memory of the second computer. 



4$£j| method accordingly to claim 1 , further comprising 
: ^l^toring in the asynchronous network interface an 
"' AX , X\on t corresponding to each triggering value, 
Is to be carried out, in the event that the trlg- 



' ^gering value is matched, to indicate that the trigger- 
;;^iiihg value has been matched. 

" ' ^method according to claim 1 , comprising the step 
pi-sending an interrupt when a triggering value 




6j^jjf method according to claim 1 , comprising the step 
;J^j|changing the value of a counter when a triggering 
^iyalue is matched. 



7*$P;£< method according to claim 1 , in which the trigger- 
if||jn]3 value(s) comprise(s) address data, and the part 
|; ^|p|'the data stream compared with the stored trig- 
! ^|^ring value(s) comprises address data. 

8^^^method according to claim 1 , wherein the step of 
.%^t or ' n 9 a triggering value is initiated by an applica- 

f^'tion on one of the computers writing a triggering val- 
.Jue to a memory location in the local control aperture 
J^vvithin the address space of the network interface. 

9^pJ method according to claim 1, comprising the 
-S^teps of accessing the main memory of the sending 
\^^^^Pplicatlon, and outputting data therein. 

1 0j^method according to claim 1 , comprising the step 
llM^f mapping each physical destination address of the 
kl8££ata being sent to a virtual memory address on a 
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sending computer. 

11. A met h od accord in g to claim 1, both computers hav- 
ing an asynchronous network interface, comprising 
the step of sending the data stream from the send- s 
ing network interface to the receiving network inter- 
face. 

1 2. A method according to claim 1 1 , comprising the step 

of mapping each virtual address of the received da- 10 
ta stream to a physical address memory location of 
the main memory of the receiving computer. 

1 3. A method according to claim 1 , comprising the step 

of writing the transferred data to the main memory is 
of the receiving computer. 

1 4. A method according to claim 1 , each computer hav- 
ing a network interface also having an I/O bus, the 
method comprising the step of providing the net- 20 
work interface with a local bus, and a bridge for in- 
terfacing between the local bus and the I/O bus of 

the computer. 

1 5. A method according to claim 2, comprising the step 25 
of loading the bridge with predetermined configura- 
tion data. 

16. An asynchronous network interface, for use in a 
host computer having a main memory and being 30 
connected to a network, the interface comprising: 

means for storing a set of rules for directing in- 
coming data to memory locations in the main 
memory of the host computer; 35 
a memory for storing or more triggering value 
(s), each value representing a state of a data 
transfer between two or more applications in 
the computer network; 

a receiver for receiving a data stream being *o 
transferred between two or more applications 
in the computer network; 
comparison means for comparing at least part 
of the data stream received by the network in- 
terface with the stored triggering values; and 45 
a memory for storing information identifying any 
matched triggering values. 




Sclaim 1 6, further comprising a local bus. 
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. n ^*j^_asynchronous network interface according to 
;,^claim 1 9, the host computer having an I/O bus, the 

f interface further comprising a bridge for interfacing 
^ ^etween the I/O bus of the computer and the local 
M$!V S of the network interface. 

2i^]An asynchronous network interface according to 
^.Clairn 16, wherein the comparison means compris- 
t^s a content-addressable memory. 

22j§£n asynchronous network interface according to 
' |j|s£' m 21 > wherein the comparison means compris- 
Jp|es two or more content-addressable memories 
•|^v/hich are arranged so as to conduct a pipelined 
'Ipcomparison of the data stream received by the net- 
i ril|wprk interface. 



2f3SM ; asynchronous network interface according to 
j:Mciaim 16, further comprising receive and transmit 
y^^l'rjalisers. 

2^1/Vn'' asynchronous network interface according to 
/ cfplaim 1 6, comprising a memory for storing configu- 
VM-raition data for the bridge. 



2%Ari asynchronous network comprising two or more 
p|computers each having an asynchronous network 



^|||nterface according to claim 1 6. 



26^-method of passing data between an application 
a f' rst computer and remote hardware within a 
^^econd computer or on a passive backplane, the 
|0ftrst computer having a main memory and an asyn- 
^Ighronous network interface, the method comprising 

(> Mfe steps of: 

<*wMf t providing the asynchronous network interface 
with a set of rules for directing incoming data to 
1^1^ memory or I/O location (s) of the remote hard- 

^IgsS? storing in the network interface one or more trlg- 
9 enn 9 va ' ue ( s )> eacn triggering value repre- 
senting a state of a data transfer between the 
application and the hardware; 
•JfSK? receiving, at the network interface, a data 



17. An asynchronous network interface according to 

claim 16, in which the set of rules comprises a mem- so j .j 
ory mapping. 



18. An asynchronous network interface according to 
claim 16, further comprising means for performing 
an action corresponding to a matched triggering 
value. 

19. An asynchronous network interface according to 



■:»>>. stream being transferred between the applica- 
tion and the hardware; 
v^r^l' comparing at least part of the data stream re- 
: V MW§ ce ' ve d w ' tn the stored triggering value(s); 
zBtffi, indicating that a triggering value has been 



matched, if any compared part of the data 
stream matches a triggering value; 
and, when a data stream is being passed from 
% . the first computer to the remote hardware, stor- 
|- ing data received by the remote hardware in 
P^' memory or I/O location(s) of the remote hard- 
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ware In accordance with the said rules; and, . 
when a data stream is being transferred from 
the remote hardware to the first computer, stor- 
ing the data received in the main memory of the 
first computer at one or more memory location 
(s) in accordance with the said rules. 

27. A method according to claim 26, in which the step 
of providing the asynchronous network interface 
with a set of rules comprises the step of establishing 
a mapping between information contained with the 
incoming data stream and one or more memory or 
I/O location(s) of the receiving computer or hard- 
ware. 

28. A method according to claim 27, in which the asyn- 
chronous network interface is a memory mapped 
network interface, and in which the step of providing 
the memory mapped network interface with a set or 
rules comprises the step of the first computer es- 
tablishing a mapping, either locally or remotely, be- 
tween addresses contained within the incoming da- 
ta stream and one or more memory or I/O location 
(s) of the receiving computer or hardware. 

29. A method according to claim 26, further comprising 
storing in the asynchronous network interface an 
action, corresponding to each triggering value, 
which is to be carried out, in the event that the trig- 
gering value is matched, to indicate that the trigger- 
ing value has been matched. 

30. A method according to claim 26, comprising the 
step of sending an interrupt when a triggering value 
matches. 

31. A method according to claim 26, comprising the 
step of changing the value of a counter when a trig- 
gering value matches, 

32. A method according to claim 26, in which the trig- 
gering vaiue(s) comprise(s) address data, and the 
part of the data stream compared with the stored 
triggering value(s) comprises address data. 

33. A method according to claim 26, wherein the step 
of storing a triggering value is initiated by an appli- 
cation on a computer writing a triggering value to a 
memory location in the local control aperture within 
the address space of the network interface. 

34. A method according to claim 26, comprising the 
steps of accessing the main memory of the applica- 
tion, and outputting data therefrom. 

35. A method according to claim 26, comprising the 
step of mapping each physical destination address 
of the data being sent, to a virtual memory address 
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l^bn a computer. 

36tvA_.method according to claim 26, both computers 
~ r %{iiaving an asynchronous network interface, corn- 



uprising the step of sending the data stream from the 

ilP| eno, ' n 9 networ ' < ' nter f ace to tne receiving network 
'^Interface. 

37$$£method according to claim 26, comprising the 
';fe§tep of mapping each virtual address of the re- 
-^ceived data stream to a physical memory address 



|i|r l/0 location of the receiving computer or remote 
':'^|ardware. 



15 ?,?{M E f. metnoc ' according to claim 26, comprising the 
V'Ppifep of writing the transferred data to the main mem- 



Jg^Ofyof the receiving computer. 



39^&method according to claim 26, each computer or 
impassive backplane having a network interface also 
|||paving an I/O bus, the method comprising the step 

*|||of providing each network interface with a local bus, 
l^^nd a bridge for interfacing between the local bus 
^grid the I/O bus of the computer or passive back- 

40>i^imethod according to claim 39, comprising the 
: |i||tep of loading a bridge with predetermined conf ig- 
4fe"uration data. 

m 

4^0£ v method according to claim 40, in which the con- 
v^figuration data includes configuration data relating 
^|c! the remote hardware. 

& t 

^method according to claim 26, each computer 
||||nd/or passive backplane having an I/O bus, the 
< _ ^jinethod further comprising the steps of: 

Mr- 

^' loading the network interface of one of the com- 
W puter(s) and/or of the passive backplane with 
f. data for configuring it to capture one or more 
predefined interrupt signal(s) on the I/O bus of 
that computer or passive backplane; 
transferring a captured interrupt signal over the 
J^p: network to a network interface of another com- 
§iSsa- puter or passive backplane; and 



loading the network interface of one of the com- 
'f|p^ puter(s) or of the passive backplane to assert 
. $&J$; one or more predefined interrupt signal(s) on 
(|^|^! the I/O bus of that computer or passive back- 

^ -tflili P' ane » on rece 'P t °f * ne sa 'd transferred cap- 
.•feM-ff tured interrupt signal. 




femethod of transferring data from a sending appll- 
<; |f^atlon on a first computer to a receiving application 
•^tg^a second computer, each computer having a 
. *i^^aln memory, and a memory mapped network in- 
^&terface, the method comprising the steps of: 
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creating a buffer in the main memory of the sec- 
ond computer for storing data being transferred 
as well as data identifying one or more pointer 
memory location(s); 

storing at said pointer memory location(s) at s 

least one write pointer and at least one read 

pointer for indicating those areas of the buffer 

available for writes and for reads; 

in dependence on the values of the WRP(s) and 

RDP(s), the sender application writing to the *o 

buffer; 

updating the value of the WRP(s), after a write 
has taken place, to update the indication of the 
area(s) of the buffer available for reads and the 
area(s) available for writes; 15 
in dependence of the values of WRP(s) and 
RDP(s), the receiver application reading from 
the buffer; and 

updating the value of the RDP(s), after a read 
has taken place, to update the indication of the 20 
area(s) of the buffer available for reads and the 
area(s) available for writes. 

44. A method according to claim 43, in which the step 

of updating the value of the WRP(s) includes the 25 
sending application sending the updated value of 
the WRP to the main memory of the second com- 
puter, via the network. 

45. A method according to claim 43, in which the first 30 
computer comprises a processing means with a 
cache memory, comprising the step of the sending 
application storing the value of the updated WRP in 

the cache memory. 

35 

46. A method according to claim 43, in which the step 
of updating the value of the RDP(s) includes the re- 
ceiving application sending the updated value of the 
RDP to the main memory of the first computer, via 

the network. *o 

47. A method according to claim 43, in which the sec- 
ond computer comprises a processing means with 
a cache memory, the method comprising the step 

of the receiving application storing the value of the 45 
updated RDP in its cache memory. 

48. A method according to claim 43, comprising the 
steps of: 



the network interface of the second computer 
storing triggering value(s) corresponding to the 
address(es) of one or more write pointer(s) 
(WRP(s)): 

the network interface of the second com- 
puter monitoring the data stream received 
from the first computer and comparing at 
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least part of the data stream with the trig- 
gering value(s) stored in its memory; and 
if any triggering value matches, indicating 
that that triggering value has matched. 



49^ ; method according to claim 48, in which when a 
■fegiSipgering value is matched by the receipt of the 
5?S-j#RP write instruction, a receiver interrupt is gener- 
m'ated. 

SQ^fA method according to claim 43, further comprising 
steps of: 

providing a second buffer in the main memory 
of the second computer for storing write pointer 
fill data; 

"^j&v stor i n 9 one or more second-buffer write pointer 
jgj£ (s) and second-buffer read pointer(s) indicating 
the areas of the second-buffer available for 
writes and reads; 

when the sending application writes to the first- 
buffer and updates the write pointer(s) of the 
first-buffer, writing to said second-buffer, in ac- 
cordance with the value of the write pointer(s) 
||ftf: and read pointer(s) of the second-buffer, the 
^" updated value of the write pointer of the first- 
buffer; and 

i0M''' updating the value of the second-buffer write 
mi pointer(s) to update the indication of the area 
C.fl^'j (s) of the second-buffer available for writes and 
the area(s) available for reads. 

51 ft&method according to claim 50, further comprising 
f^ttjp steps of: 

$J reading a first-buffer write pointer value from 
the second buffer, in dependence on the con- 
^jM) tents of the second-buffer read pointer(s) and 
f||§j$ second buffer write pointer(s), and 



reading from the first buffer in dependence on 
i^P^fe tne va,ue °f a first-buffer pointer and the write 
l Bll' pointer value read from the second buffer. 

S^^^nethod according to claim 43, further comprising 
'"^tfie steps of: 



55 



the network interface of the first computer stor- 
' n 9 triggering value(s) corresponding to ad- 
; fel^: dress (es) of one or more RDP(s); 
^WM$ the network interface of the first computer mon- 



itoring the data stream received from the sec- 
• ond computer and comparing at least part of the 

, data stream with the triggering value(s) stored 

In its memory; and 
J§S;| If any triggering value matches, indicating that 

that triggering value has matched. 

53^A!method according to claim 52, in which when the 
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network interface of the first computer matches a 
triggering value by the receipt of an RDP write in- 
struction, a sender interrupt is generated. 



fsjj available for reads; 
& the network interface of the second computer 



scgmprising: 



54. A method according to claim 43, in which the send- 
ing application blocks if the values of the WRP(s) 
and RDP(s) indicate that the buffer is full. 

55. A method according to claim 54, in which the send- 
ing application is unblocked on receipt of an inter- 
rupt. 

56. A method according to claim 43, in which the receiv- 
ing application blocks if the values of the WRP(s) 
and RDP(s) indicate that the buffer is empty. 

57. A method according to claim 56, in which the receiv- 
ing application is unblocked on receipt of an inter- 
rupt. 

58. A method according to claim 43, in which a write 
pointer of a buffer points to the buffer address where 
the next byte of data should be written in that buffer. 

59. A method according to claim 43, in which a read 
pointer of a buffer points to the buffer address of the 
first byte of data to be read from that buffer. 
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a memory mapping; 

means for reading data from the buffer in ac- 
cordance with the contents of the WRP(s) and 
RDP(s); and 

means for updating the value of the RDP(s) af- 
. ter a read has taken place, to update the indi- 
cation of the area(s) of the buffer available for 
c^lpffi. reads and the area(s) available for writes. 

64?£Axomputer network according to claim 63, the net- 
work interface of the first computer comprising: 

a mapping memory; and 
means for sending data to the buffer of the sec- 
ond computer. 



65ip^'computer network according to claim 63, the main 
&|nemory of the second computer storing the value 
' p|bi'at least one WRP. 



66'i^computer network according to claim 63, in which 
j§>ne or more pointer memory location(s) are in the 
naln memory of 'the first computer. 



60. A method according to claim 43, in which when an 
application has written to the end of a buffer, it next 
writes to the start of the buffer, depending on the 
value of the WRP(s) and RDP(s) corresponding to 
that buffer. 

61. A method according to claim 43, in which when an 
application has read to the end of a buffer, it next 
reads from the start of the buffer, depending on the 
value of the WRP(s) and RDP(s) corresponding to 
that buffer. 

62. A method according to claim 43, in which the value 
of one or more WRP's and/or RDPs is updated 
when a triggering value is matched in a network in- 
terface. 

63. A computer network comprising two computers, the 
first computer running a sending application and the 
second computer running a receiving application, 
each computer having a main memory and a mem- 
ory mapped network interface, the main memory of 
the second computer having: a buff er for storing da- 
ta being transferred between computers as well as 
data identifying one or more pointer memory loca- 
tion^); 

means for reading at least one write pointer 
(WRP) and at least one read pointer (RDP) stored 
at (a) pointer memory location(s), for Indicating the 
areas of the buffer available for writes and the area 
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67^A : computer network according to claim 63, in which 
:»j»ne or more pointer memory location (s) are located 
jfeffrr the main memory of the second computer. 

68MA computer network according to claim 63, in which 
^fe^ie first computer comprises a processing means 
!*$lt$rto a cache memory, with one or more WRP(s) 
• ( M^|ci/or RDP(s) stored in that cache memory. 

69§v|computer network according to claim 63, in which 
1 ' fffl? second computer has a processing means with 
''ijrcache memory, with one or more WRP(s) and/or 
§|$pP(s) stored in that cache memory. 

fill' , 

70|^ : ( computer network according to claim 63, in which 
-Bf&ie network interface of the first computer compris- 



means for writing data to the buffer in accord- 
ance with the values of at least one RDP and 
one WRP, using its memory mapping; and 
means for updating the value of the WRP(s) to 
|||& update the indication of the area(s) of the buffer 
. available for reads and the area(s) available for 
Iff writes. 



55 yjl^A^computer network according to claim 63, in which 
^^^le'main memory of the second computer compris- 
' a second buffer; and the computer network also 
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means for reading one or more write pointer(s) 
and one or more read pointer(s) of the second 
buffer indicating the areas of the second buffer 
available for writes and those available for 
reads; 5 
means for updating the write pointers) of the 
first buffer, when an application running on one 
of the computers writes to the first buffer; 
means for writing to said second buffer, in ac- 
cordance with the value of the write pointer(s) '0 
and read pointer(s) of the second buffer, the up- 
dated value of the write pointer of the first buff- 
er; and 

means for updating the value of the second 
buffer's write pointer(s) to update the indication 15 
of the area(s) of the second buffer available for 
reads and the area(s) available for writes. 

72. A computer network according to claim 71 , further 
comprising means for storing one or more write 20 
pointer(s) of the second buffer indicating the areas 

of the second buffer available for reads and the area 
(s) available for writes. 

73. A computer network according to claim 63, in which 25 
the first and/or second buffer in a circular buffer. 
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comparing at least part of the data stream being 
transferred with the stored triggering value(s); 
and 

means for indicating that a triggering value has 
been matched, when the part of the data stream 
being compared matches a triggering value. 

7§I^ omputer network ac °ording t0 c ' aim 75 > in which 
' G&tn e means for indicating that a triggering value has 
J^een matched comprises means for generating an 
'^Interrupt. 

77§£;method °* transferr,n 9 data t0 a buffer of a receiv- 
-jr^|fng computer application, comprising the steps of: 

Iff 

storing in a sending application a write pointer 



representing the position of the start of a first 
: |pif section of the buffer available for receiving da- 
ta; 

; " storin 9 in tne receivin 9 application a copy of the 
write pointer; 

sending a first block of data to the first section; 

.lit and 

updating the write pointer in the sending appli- 
cation and the copy of the write pointer in the 
receiving application. 



74. A computer network according to claim 71 , in which 
the network interface of the second computer also 
comprises: 30 

means for reading a first-buffer WRP value 
from the second buffer in accordance with the 
values of the second-buffer WRP(s) and RDP 
(s); 35 
means for updating the RDP(s) of the second 
buffer to update the indication of the areas of 
the second buffer available for reads and 
writes; 

means for reading from the first buffer in ac- *o 
cordance with the contents of the first-buffer 
WRP value read from the second buffer, and a 
first-buffer RDP; and 

means for updating the value of the RDP(s) of 
the first buffer to update the indication of the 45 
area(s) of the first buffer available for reads and 
writes when an application running on the sec- 
ond computer reads from the first buffer. 

75. A computer network according to claim 63, the net- 50 
work interface of one or both computers also com- 
prising: 

fa memory for storing triggering value(s), cor- 
responding to one or more address(es) of WRP 55 
(s) and/or RDP(s); 

means for monitoring a data stream being 
transferred between the two computers and for 



78.^$ method as claimed in claim 77, comprising the 
l||teps of: 

ft'! 

' fep-f storing in the receiving application a read point- 
rmm; er representing the position of the start of a sec- 
^Mfe' ! ond section °f tne buffer containing data avail- 
able for reading; 

storing in the sending application a copy of the 
read pointer; 

reading in the receiving application a second 
block of data from the second section; and 
updating the read pointer in the receiving appli- 
cation and the copy of the read pointer in the 
sending application. 

79|^|nethod as claimed in claim 78, comprising the 



comparing in the sending application the write 
pointer and the copy of the read pointer to de- 
termine the size of the first section; and 
sending the first block of data which is no bigger 
than the size of the first section. 



8^i^;method as claimed in claim 78, comprising the 
^jsteps of: 

comparing in the receiving application the read 
pointer and the copy of the write pointer to de- 
termine the size of the section; and 
reading in the receiving application from the 
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second section the second block of data which -i 
is no bigger than the size of the second section. i 

81 . A method as claimed in claim 77, in which the first V u ja&1 



application is on a first computer and the second 
application is on a second computer separated from 
the first computer by a network. 



block of data is sent from the sending computer ap- s 
plication. 

82. A method as claimed in claim 77, in which the buffer ' Jjp^- 
is a circular buffer. ; 

10 m 

83. A method as claimed in claim 77, in which the first 



15 




CNl 

mo 



c 



o 

Q) 
C 

o ~a 

CD 
Q_ 
CL 
O 

E 

>s 
i_ 

o 
E 

CD 



0/21/05 




o 

CM 
0) 
Q_ 

E 
o 
o 



r 



i 

cm I' E 

CN 

cn 

c i w 



O 0) 

c ° 

tOi 



si 





0) 




_c 




o 




o 


Z) 


o 


CL 




o 






Q.| 2 

< I CL 
O 



o 
E 

cu 




10/21/05, 



EP 



1 302 853 7 A# 



CD 

u 
o 
O 



o 
E 

CD 



Z> 
CL 
O 



CD 














u 






o 






o 














lO 








CL 




O 


















C CD 






CD 











CD 



o 

CD 



LO 

Z) 

a. 

o 



v_ 
CD 

" c 





00 




LO 




\_ 


CD 




V) 


o 
\_ 






CO 


c 




o 




a 





00 






E 




CD 


~o 


(/) 






c 


in 


O 




U 



p5£ 



O. 



CD 



o, 



CO 
CD 

O 

c 



CD 

O 




CD O 
-Q £ 



<D 
C 



CO 
CD 
T3 
O 
C 



CD 



LO 



C CD 
CD-*-* 
> 3 
C Q_ 

E 

C° 
5° 
(/) „ 
CD O 

CL-+-' 

a 

H — =5 

w c 
O o 
^ o 



c 

CO J 



10/21/05, EAST Version: 



2.0.1 .4 







O 




CL 




LU 


D 


O 


_Q 


LO 


CD 

o cn 




> 


U XI 




o *c 

7-° 


em 


o 


CO 


CL 


V3 



C/) 

o 

CL 



C 

i c 

(D 

v+- > 
O C 



> 






CM 


\ 






d> 




M— 
M— 


cn 


.3 










GO 


O 


O 


O 




o 





.-9 3*fc 



10/21/05, EAST 




CO 

o: 
_o 

_Q * 



CD 



CD 



2c 

o v 

4 

4- XI 

E 



CD 







o 








o 






N 


CD 


X 


gen 


m 




CD 


o 




o 




o 





(D 
CT»C 
C O 

|.E 

goo 



s 



m 

C\! 



2.0.1 .4 




0/2 1 /O 5 , EAST Ver^iSp : 



'IP* 

EP 1 302 853^: 



0) 



on 

< ^ 

CD CO 
< 



co 
o 



CO 



O 

CD D 
CD 



CO 
CO 
CD 

XI 
< 



co CD 

c 
Cr» O 

C ' 



o 

CO 



o 
u 



o 

u — 
c 

■So 

H O W 
c >> 

E ^ 

CD 

E -Q 



QJ D 



Q) 

o 00 



p 

D 

X CO 
UJ - 

JO 



c 

CD 

E 

O 
_Q 

E 

CD 

~o 

U 
O 

o 

Q_ 



C 



\ 


o 




Burst 
address 


(iny^|d) 


Address 
parity 


$ 

\ 


o 


o 


1st 

data item 


1st bytejlbhable 


1st 

data parity 


r/w 


o 


o 


2nd 
data item . 


■ 2nd * y i§ abie 


2nd 
data parity 














r/w 


o 


o 


n-lth 
data item 


n-1 tn 'by$g*;£enable 


n-lth 
data parity 


\ 




o 


n th 
data item 


nth bytQlfnable 

• ■' 


n th 

data parity 


3 bits 


32 bits 


m 


4 bits 






•IP 

43 bitsSl 




mm;- 



CD 



E 




10/21/05, EAST Version: 2.0.1 



■m 

EP 1 302 853 $2. 



^ ^ ° 

< < < 

O O _J 

c c U- 



o 

CF) 



o 



r 



A » 



o 




o 
c 


{n/a f n/a,0,1] 


A fi s(31:l6) 




o 


T3 


Address parity 


Affss(15:0) 


\ — 


o 




1 st byte enable 






o 


M — 


1st data parity 


1 s ||§ata(15:0) 




o 




2 nd byte enable 


2 n lSata(31:16) 




o 


X> 

cc 


2 nd data parity 


2#data(15:0) 






o 






n— 1^ data item 






o 


Rfd 


n^ h data item 






o 


o 


n-1^ data item 






o 


X) 

q: 


n^ 1 data item 


3 bits 


4 bits 




n -w ta(51:16) 



n-^dato(15:0) 



nt||gta(31:l6) 



15:0) 



:;gl6 bits 



L 



23 bits 



0> OT 

E jq 

o 

x 

Ll) 



CD 

E 



C 

(D 

o 



10/21/05, EAST Versfdp: 2.0.1.4 



> > o 

< < < 

O O -J 

C C Li- 



r 




o 




o 

\ 

c 


{n/a.n/a.O.O} 


A|pss(31:16) 




o 


~o 
cr 


Address parity 


AJpress(1 5:0) 




o 




1 st byte enable 


1|§M31:16) 




o 


X) 

H— 

cr 


1st data parity 


1.Sdta(15:0) 




o 




2 nd byte enable 


2@iata(31:16) 




o 


cr 


2 nd data parity 


^pjata(15:0) 










••ftp' 

w \ 

w 




o 




data item 


n-^Jata(31:16) 




o 


"D 

s— 

cr 


n^ 1 data item 


n-H| data(15:0) 




o 


o 


n-l^h data item 


n|§pata(31:16) 




o 


"D 

cr 


n^ 1 data item 


J|data(15:0) 


3 bits 


4 bits 


bits 




23 bits 


'. IWwW m 



.2 o 

Cl 

O Ll_ 

t o 

^ a I 
E o x 



E -Q 
o 

X 
LJ 



(1) 

E 
i- 



\_ 
Z5 
.O 

■a 
o 

CD 

cr 
c 

CD 

E 
o 

_Q 

E 

CD 
O 

o. 
o 

o 

CL 



c 

CD 

o 



10/21/05, EAST Ver^n: 2.0.1.4 





o 




1 st byte enable 


^dato(31:16) 




o 


T3 

a: 


1st data parity 


data(15:0) 




o 




2nd byte enable 


•ggdata(31:16) 




o 


H— 

cr 


2 nd data parity 


'■g|ddata(15:0) 





o 




n-1 tln data item 


r^ h dato(-31:16) 




o 


Rfd 


data item 


|||| h data(15:0) 




o 


o 


n— 1 ^ data Item 


|||data(31:16) 




o 


Rfd 


data item 


J|hdata(15:0) 


3 bits g I 


4 bits 


a is bits 




23 bits 


•"Si .. — - 




E -Q 
o 

X 



EP 1 302 853 A2 



^ ^ «- 

O Q -J 



r 




o 




o 

\ 

c 


[n/a.n/a,! ,1 } 


Esf§|ata(31:16) 




o 


"D 

cr 


Address parity 


EsJ|tdata(l5:0) 




o 




V s t byte enable 


1§ta(31:16) 




o 


cr 


1st data parity 


1 : fgata(15:0) 




o 




2 nd byte enable 


2^bta(31:16) 




o 


cc 


2 nd data parity 


2g||data(l5:0) 










* 'life 

i mm- 
! Wt 




o 




n-1^ data item 


nf|Sata(31:16) 

■Itfe 




o 


"O 


n^ data item 


n^»data(15:0) 




o 


o 


n — 1^ data item 


n||fdata(31:16) 




o 


(T 


data item 




3 bits 


4 bits 


fll-6 bits •' 




23 bits 



mr 



> > o 

< < < 

O O — I 

c c U- 



r 



o o : 
O cn ! » 



o 




o 

\ 

c 


{n/a,n/a,1,0} 


E S atQ(31:16) 




o 


T> 

4— 

a: 


Address parity 


Etedata(15:0) 

■few?. 


3 bits 


4 bits 


«W£ bits 




23 bits 





CD 

E 



CD 



O 

a 
o 

o 

CL 

-+"-* 
C 

CU 
l_ 
i_ 

o 



•II 
II 

10/21/05, EAST Version: 2.0.1.4 



m 

EP 1 302 853/A2 




10/21/05, EAST Version : 2.0.1.4 



Computer 



ROM 



^305 



31 1 



Content 
addressable 
memory 

(CAM) 



307 



.m 



-IML 



^-306 

End— point 
application 



Code 
generator 



^309 
^-310 



Action 
generator 



•308 



Memory 



1 



I 



Network|M)3 



Computer 



ROM 



-305 



31 1 



Content 
addressable 
memory 

(CAM) 



Process 



307 



r End-point 
' application 



1 




W0: 



EP 1 302 853 AS:, 



310 





'Raise IR(i 



"Hi; 



it 



320 




FIG 1 8 |||i. 




EP 1 302 853: J A2 



^310 



Action 
generator 



Write' N'to'iSp- 

•Mm 



10 



Action 
generator 



m 

•Mi 



1PP 



FIG 



2ft 



^•308 



Memory 
^-323 




N 








Loc n X 






^308 



Memory 



1 



3J4 





FIG 23» 




1 0/21/05, 




2.0.1 .4 



Implied addresses^f- 






n + p-1 




n+3 


n+2 




n 




Dp 




D 4 


D 3 


m 




A(n) 



Data burst 



FIG 



330 

S 



3i'-'> 
!6 



332 


Forwarding 
unit 






D n + 1 


Win A(n) 








•afSF — 




11 



330 






Forwarding 
unit 


,#|b33 
:W ) 




~m °n + 1 A(n + 1) 




:llf 







1 0/21/05, 




EAST Version : 2 . 0 . 



1 .4 



Second burst 
337 



M; : - 

EP 1 302 853 ;A2 

m 



First burst 
336 



D 



n + 1 



An + 1'«H 



n 



\ as* 

Internol buffe# 
335 ;f§ 




CoalescedMIS' 

________ W-m;i.< 



IfSjoalesced burst 

338 



:.m 






D nll 











Internal buffjp 
335 



FIG 27 





EP 1 302 853^A2/ 




MgJ|>ed configuration 
dMq and registers 
o^cemote hardware 

IP' 342 



iress mapping on to 
/Jfpisystem I/O Bus. 



jjNIC Will raise interrupts 
and write data on 
behalf of remote 
hardware. 



fretwork card has been 
[loaded with configuration 
J * and register data 
corresponding to 
remote hardware 



A 



Con figuration and registers 
of remote hardware 
341 



Data for 
application 



Remote hardware 
receives requests 
from application 

and may also 
raise an interrupt 

or write data 
back to application 



^ {Remote hardware 

*~ 341 

° FIG 28 



10/21/05, EAST Version: 2.0.1.4 



EP1 302 853^2 



7» 



Pf 

Computer 00 



Tripwire 
unit 4 



Tripwire 
unit 3 



Tripwire 
unit 2 

351 




Tripwire 
unit 5 



Tripwire 
unit 1 

350 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 

BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

^ BLACK BORDERS 

IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 
$ FADED TEXT OR DRAWING 

□ BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

(Jjf COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



